-- Logs begin at Mon 2022-08-29 18:12:54 UTC, end at Mon 2022-08-29 19:48:08 UTC. -- Aug 29 18:57:05.744305 dv-cinder-855013-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 18:57:05.751596 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 18:57:05 2022] *** Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: nodename: dv-cinder-855013-1-iscsi Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: machine: x86_64 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: clock source: unix Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: pcre jit disabled Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: detected number of CPU cores: 4 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: current working directory: / Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: detected binary path: /usr/bin/uwsgi-core Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: your processes number limit is 47680 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: your memory page size is 4096 bytes Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: detected max file descriptor number: 2048 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: lock engine: pthread robust mutexes Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: thunder lock: enabled Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 18:57:05.758786 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 18:57:05.798619 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: Python main interpreter initialized at 0x56291897c9c0 Aug 29 18:57:05.798619 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: python threads support enabled Aug 29 18:57:05.798619 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: your server socket listen backlog is limited to 100 connections Aug 29 18:57:05.798619 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: your mercy for graceful operations on workers is 90 seconds Aug 29 18:57:05.798972 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 18:57:05.799056 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: *** Operational MODE: preforking *** Aug 29 18:57:05.799302 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: *** uWSGI is running in multiple interpreter mode *** Aug 29 18:57:05.799302 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: spawned uWSGI master process (pid: 81876) Aug 29 18:57:05.799512 dv-cinder-855013-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 18:57:05.802982 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: spawned uWSGI worker 1 (pid: 81890, cores: 1) Aug 29 18:57:05.802982 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: spawned uWSGI worker 2 (pid: 81891, cores: 1) Aug 29 18:57:05.802982 dv-cinder-855013-1-iscsi devstack@keystone.service[81876]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 18:57:10.150073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:57:10.150073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: from cryptography.utils import int_from_bytes Aug 29 18:57:10.150839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:57:10.150839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: from cryptography.utils import int_from_bytes Aug 29 18:57:10.151250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:57:10.151250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: from cryptography.utils import int_from_bytes Aug 29 18:57:10.151699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:57:10.151699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: from cryptography.utils import int_from_bytes Aug 29 18:57:10.200493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:57:10.200685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:57:10.200875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] command line args: [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:57:10.201141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:57:10.201317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ================================================================================ {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:57:10.201568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] admin_token = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.201812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] config_dir = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.201997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.202177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] config_source = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.202346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:57:10.202427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] debug = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.202539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.202798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:57:10.202798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] command line args: [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:57:10.202920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:57:10.203148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ================================================================================ {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:57:10.203285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] admin_token = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.203829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] config_dir = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_config_append = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] config_source = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_dir = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] debug = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_file = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.204930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_config_append = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.205958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_dir = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_file = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.206903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.207696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.208314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.208499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.208760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.208855 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.208967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] publish_errors = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.209927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] standard_threads = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.210909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] publish_errors = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] use_journal = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] use_json = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] use_stderr = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] use_syslog = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.211825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.212070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.212150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.212308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.212397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.212555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] standard_threads = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.212632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.212790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.212874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.213038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.213124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.213223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.213301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.213424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] use_journal = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.213501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.213646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] use_json = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.213720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.213795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] use_stderr = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.213934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] use_syslog = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.214011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:57:10.214287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.214928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.215998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.216974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.217960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.218874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.219923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.220919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.221921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.222861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.223937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.224852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.225563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.226931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.227909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.228955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.229895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.230992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.231793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.232905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.233899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.234853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.235951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.236963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.237939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.external = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.238939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.239543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.239636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.239711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.239821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.239949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.password = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.240071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] auth.token = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.240149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.247607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.247759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.247877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.247955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.external = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.248930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.password = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] auth.token = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.249964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.250935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.251982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.252905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.253980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.254776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.255982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.256903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.257986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.258953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.259928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.260960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.261902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.262953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.263922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.264881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.265974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.266927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.267920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.268996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.269951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.270954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] role.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.271907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.272075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] role.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.272156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.272308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.274997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.275927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] role.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] role.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.276997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.277970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.278912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.279964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.280971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.281958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.282924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.283920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.connection = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.284912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.285853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.286977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.287953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.288979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.289059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81890) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:57:10.289208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.289428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.289692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.connection = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.290028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.290288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.290558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.290811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.291046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.291288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.291529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.291770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.292050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.292294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.292542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.292780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.293030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.293276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.293525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.293759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.294003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.294241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:57:10.294413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81891) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:57:10.321235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.321457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.321677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.324386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.324632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.324860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.330121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.333441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.364534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.364760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.365869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.366109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.436256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.436873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.488157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.488477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.489932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.490150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.650827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.651078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.651372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.651659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.651960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.652855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.653070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.676481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.677829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.678769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.679046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.689716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.690206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.710906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711781 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.711884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.712110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.712203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.712328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81891) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.712496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.712788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.713064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81890) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:57:10.729710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.729824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.729903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:57:10.730082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:57:10.730247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.730448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:57:10.730583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.730744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.730891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:57:10.731010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.731274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.731364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.731654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.731794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.732057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.732384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.741335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.741614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.741887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.742173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.742536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.742897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.743292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.743644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.749309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.750795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.752246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.752529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.752811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.753091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.753370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.753630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.753769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.753925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.754005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.754162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.754243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.754524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.755109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.755336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.755445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.755609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.755712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.756014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.756292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.756564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.756839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.757106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.779999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.781475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.781758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.782365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.783822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.784123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.786512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.786799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.787086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.788836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.789118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.789404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.793510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.793797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.795990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.796286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.797905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.798198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.798488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.798776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.799056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.799369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.799687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.800006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.800272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.800564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.800853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.801235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.801544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.803676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.803998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.804290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.817914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.820693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.820786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.820983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.822106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.824925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.825369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.825671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.828103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.828391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.829911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.833412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.833671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.836590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.836924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.837226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.837516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.837815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.838106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.838412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.839723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.840048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.840337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.840620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.840908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.841290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.854517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.854804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.855077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.855356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.857701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.857987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.858263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.858530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.862302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.863840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.865355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.865504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.865710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.867067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.868414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.868707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.868804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.869054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.869136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.869395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.869695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.871690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.872008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.872309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.872606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.872896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.881578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.881864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.882246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.882533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.882821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.883104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.883395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.885253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.885253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.885721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.885883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.886168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.886448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.886746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.900809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.902894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.903771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.905877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.906715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.908188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.908774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.909807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.910093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.910225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.910376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.911720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.912031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.912323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.916839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.918323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.918993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.920544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.927463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.927757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.930126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.930423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.930706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.932527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.932818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.933102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.938157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81891) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.938442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.938726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.939006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.939283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.939556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.939878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81890) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:57:10.940185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.940876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81891) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.941980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.942797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.943077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.943349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.943612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.943899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.944178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81890) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:57:10.977765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56291897c9c0 pid: 81891 (default app) Aug 29 18:57:10.979226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56291897c9c0 pid: 81890 (default app) Aug 29 18:57:10.981410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ed283-42c7-446c-90ca-76203d945908 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:10.981730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ed283-42c7-446c-90ca-76203d945908 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:10.982003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ed283-42c7-446c-90ca-76203d945908 None None] PATH_INFO: `/v3` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:10.986190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Mon Aug 29 18:57:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:18.343846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:18.344370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:18.344370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:18.348882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:18.353963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:18.354372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:18.354492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:18.423628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: INFO dbcounter [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] Registered counter for database keystone Aug 29 18:57:18.442783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG oslo_db.sqlalchemy.engines [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81891) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:57:18.448932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writer thread running {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:57:18.632232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG passlib.handlers.bcrypt [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81891) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:57:18.632556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG passlib.handlers.bcrypt [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81891) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:57:18.693901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:18.695492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:18.826090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 18:57:19.293409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:19.361985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: INFO dbcounter [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] Registered counter for database keystone Aug 29 18:57:19.383827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG oslo_db.sqlalchemy.engines [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81890) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:57:19.390015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writer thread running {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:57:19.508859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf9586-d004-4782-9a74-753165895c8e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:19.513208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:19.513821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:19.514100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:19.514371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:19.522445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:19.573663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING keystone.common.rbac_enforcer.enforcer [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:57:19.573955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db72c-afae-42f5-8310-c4e076e9c91a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:19.583336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Mon Aug 29 18:57:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 18:57:19.589378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:19.604451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c9be1-04f7-4e75-a2df-cea87d2aec34 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:19.608894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:19.610010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:19.610292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:19.610565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] PATH_INFO: `/v3/projects/bb936e104dc24a75a45f4d04495d191b` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:19.612169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] RBAC: Authorizing `identity:get_project(project_id=bb936e104dc24a75a45f4d04495d191b)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:19.664705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:57:19.665051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a14b6-06f3-45ed-aded-8757ff6afb49 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:19.669576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Mon Aug 29 18:57:19 2022] GET /identity/v3/projects/bb936e104dc24a75a45f4d04495d191b => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:21.557018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a78ee-409d-426e-b277-7fd7bf511e44 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:21.557532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a78ee-409d-426e-b277-7fd7bf511e44 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:21.557532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a78ee-409d-426e-b277-7fd7bf511e44 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:21.561645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:21.567878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:21.568317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:21.568716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:21.593075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:21.594681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:21.602837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 18:57:21.872848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a78ee-409d-426e-b277-7fd7bf511e44 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:21.885051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a78ee-409d-426e-b277-7fd7bf511e44 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:21.889259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:21.889854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:21.890134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:21.890404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:21.898140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:21.899952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d380a-664f-4696-b480-c64959943b59 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:21.908656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Mon Aug 29 18:57:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 18:57:21.914452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:21.923437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321d74c-de20-4a74-a6e3-8b75aa332f82 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:21.928385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:21.929006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:21.929287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:21.929563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:21.930872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:21.932742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c17c7-08a8-4448-adbf-e5940a9b14b8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:21.934249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Mon Aug 29 18:57:21 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:23.912819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04070-cdbf-46aa-a5cd-107e49a44dff None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:23.913331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04070-cdbf-46aa-a5cd-107e49a44dff None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:23.913417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04070-cdbf-46aa-a5cd-107e49a44dff None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:23.914797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:23.919976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:23.920405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:23.920652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:23.939122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:23.940429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:23.986514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4dea-e3aa-47b7-bbc6-4cead90af927 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:23.986842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4dea-e3aa-47b7-bbc6-4cead90af927 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:23.987065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4dea-e3aa-47b7-bbc6-4cead90af927 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:23.988605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:23.993893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:23.994172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:23.994438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.023049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:24.036789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG passlib.handlers.bcrypt [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81890) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:57:24.037181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG passlib.handlers.bcrypt [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81890) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:57:24.086538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:24.088124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:24.186697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:24.304750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.328479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:24.407063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7da3a-9b4b-4389-abed-22436c1f6d96 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.440811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.441376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.441680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.441955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.452407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.453536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:24.462610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:24.463654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:24.464809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:24.469359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.549365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:24.556834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bdc48-603c-42b9-8fa5-9469d3703b0a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.559282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:24.566864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e174-2e57-4c76-8ce6-95e0f260e410 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.594232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.595209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.595493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.595769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.600279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.601458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.602261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.602534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.603200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:24.603502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:24.604938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:24.622024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:24.654081 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.665155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:24.666915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:24.741783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ce010-2615-45e3-a378-55fe68459664 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.759508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.760175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.760462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.760732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.761880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:24.763353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:24.764977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 9/16] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:24.771003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.773884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 8/17] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:57:24.779454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.782740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:24.788291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:24.874813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cab3b-2ff9-45aa-bfba-2188f038df89 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.876213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad1a11-ab53-4b72-be9f-c94a87478ddd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:24.909536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.910414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.910536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:24.910695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.910969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.911124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:24.911400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:24.911665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:24.912814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:24.914255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:24.915789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:24.921528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:24.929521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:24.931102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:24.949572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:24.951399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:24.952952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:24.952952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:24.952952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:24.952952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:24.956481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:25.012337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:25.014715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:25.030572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba56ff-fd56-486e-976b-f09b6bd5a664 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:25.063137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e2393-1792-4425-81ee-03ea4fc849ab None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:25.065386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:25.065730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:25.066034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:25.066310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:25.067558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.domain.id=default, user.id=417d5111f71945cab4c66bd4637fb5fb) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:25.067876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:25.069335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf5dbb-d2d6-408b-aff9-ef70dbde2381 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:25.075992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/role_assignments?user.id=417d5111f71945cab4c66bd4637fb5fb&scope.domain.id=default&role.id=ae077013d56245c7869e76158c81fd53 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:25.099791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:25.100466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:25.100749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:25.101029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:25.101706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:25.102011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:25.103459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4219-bb7b-4080-822d-58630c4cc9a4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:25.123197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:27.132315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53385d-3d58-46e9-892f-468a4b615047 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.132851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53385d-3d58-46e9-892f-468a4b615047 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.132851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53385d-3d58-46e9-892f-468a4b615047 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.134185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:27.139961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930346-0b7b-4c8f-adc4-8c9c4267c4d4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.140374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930346-0b7b-4c8f-adc4-8c9c4267c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.140544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930346-0b7b-4c8f-adc4-8c9c4267c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.172559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1a930346-0b7b-4c8f-adc4-8c9c4267c4d4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:27.173873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1a930346-0b7b-4c8f-adc4-8c9c4267c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:27.268707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:27.554395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1bbb9-55c6-429c-ae53-d1ed1fe68721 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.554770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1bbb9-55c6-429c-ae53-d1ed1fe68721 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.554898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1bbb9-55c6-429c-ae53-d1ed1fe68721 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.556327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:27.564780 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.565070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.565350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.585717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1225d60-3f14-49f5-8233-05ba4d0f285a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.585965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1225d60-3f14-49f5-8233-05ba4d0f285a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.586393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1225d60-3f14-49f5-8233-05ba4d0f285a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.587873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 14/25] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:27.599066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:27.600297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:27.603781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.604267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.604689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:27.640922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:27.642386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:27.669117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:27.706449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 12/26] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:27.780571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 15/27] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:27.812424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:27.935168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcc7dd-320c-47fb-b591-9f66f3916561 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:27.994226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:27.994879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:27.995163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:27.995434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.013072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:28.020125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:28.020931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:28.021689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:28.024283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 13/28] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:28.029757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.060545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.129050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:28.210298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:28.240789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb0de-17aa-4379-b180-9a42192cb03a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.248702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba9e55-fc1c-4867-a2ce-a1b4416c65f7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.277371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.278373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.278856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.279330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.291866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.293387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.294985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.295656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.295967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.296236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.297079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:28.297546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.299069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.308767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 16/29] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:28.314098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-3657f09a-8b76-46d7-94dd-8db8a5244c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:28.314230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:28.315188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:28.316255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:28.316255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:28.316255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:28.317332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 14/30] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:28.320828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539925-7035-419c-a29c-b21965d0ed13 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.320828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539925-7035-419c-a29c-b21965d0ed13 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.320828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539925-7035-419c-a29c-b21965d0ed13 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.320828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 15/31] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:28.321688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539925-7035-419c-a29c-b21965d0ed13 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.420957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:28.434583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeebb8-2e59-42d9-ab1d-cde7f6dd366f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.438841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539925-7035-419c-a29c-b21965d0ed13 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.472363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.474195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.474636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.474818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.475232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.475613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.476084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.476427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.477374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.477575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.479078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.479317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e997fc-4c6d-4f03-a486-5ffb29ffe510 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.481389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 16/32] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:28 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:28.482754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.483184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.483585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.492235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:28.503370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:28.506348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:28.582003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 17/33] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:28.582003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.738954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 17/34] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Aug 29 18:57:28 2022] POST /identity/v3/roles => generated 219 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:28.739922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.773231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:28.807935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f897-d9cc-4ca7-9536-d99b32ac8c5e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.864274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e965-6a4b-43d9-b565-7c9d5fabbe04 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:28.899651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.901031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.901538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.901813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.903721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:28.904124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.906975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.915702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:28.917036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:28.917314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:28.917588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:28.921332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:28.926682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:28.928473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 18/35] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:28.932619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 18/36] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:28.936918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:28.938146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:29.012156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:29.054466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d834165-8d46-43e7-ab47-99fcf88c1772 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.059616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483ee30-8b7c-49bd-9681-163582b94d0f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.090542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.091158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.091440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.091717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.092468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:29.093848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ef80a-364a-463c-9049-1addde2bf4c2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:29.095688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.096381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.096665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.096971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] PATH_INFO: `/v3/domains/default/users/417d5111f71945cab4c66bd4637fb5fb/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.099514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=417d5111f71945cab4c66bd4637fb5fb, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:29.101424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:29.188288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 19/37] 10.222.0.62 () {58 vars in 1361 bytes} [Mon Aug 29 18:57:28 2022] PUT /identity/v3/domains/default/users/417d5111f71945cab4c66bd4637fb5fb/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:29.189084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:29.272850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 19/38] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Aug 29 18:57:28 2022] POST /identity/v3/roles => generated 225 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:29.274432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777ec8-b490-4c67-acf9-6dccecb11798 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.274721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777ec8-b490-4c67-acf9-6dccecb11798 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.275000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777ec8-b490-4c67-acf9-6dccecb11798 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.276861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 20/39] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:29.278198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd2501-40a6-4e89-883f-9ac21cd6fb20 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.278481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd2501-40a6-4e89-883f-9ac21cd6fb20 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.278756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd2501-40a6-4e89-883f-9ac21cd6fb20 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.280042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 21/40] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:29.281830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4260cd-0396-4fd9-9f38-d6454c91ffd6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.282223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4260cd-0396-4fd9-9f38-d6454c91ffd6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.282590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4260cd-0396-4fd9-9f38-d6454c91ffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.304606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4a4260cd-0396-4fd9-9f38-d6454c91ffd6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:29.306481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4a4260cd-0396-4fd9-9f38-d6454c91ffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:29.315489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:29.328236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f49c-582d-4f85-909a-6e5a55da3b18 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.365860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:29.384109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.384875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.385195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.385490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.386240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:29.387880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:29.393941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 22/41] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:29.395250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.395639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.396010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.428690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:29.430021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:29.518176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 23/42] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:29.547553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 20/43] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:57:29 2022] POST /identity/v3/projects => generated 301 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:29.691952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:29.762065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:29.797986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9bd8-e898-4b82-982d-5ebd5f50961b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.815069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:29.839182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.839820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.840125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.840399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.841520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:29.843149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:29.844933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 24/44] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:29.852496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:29.922960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9056-3a66-4ba9-be64-507ce156b79d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.955356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d9b2-9930-47ce-9d38-d5fa435f6089 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:29.961326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.961938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.962220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.962898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.963586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:29.965407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12509119-102f-47cb-8587-e2a31f50c712 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:29.997494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:29.998145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:29.998441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:29.998734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:29.999434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:30.000998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f718e78-80ce-48dc-8535-95a296a4e7ee None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:30.006606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:30.029290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 21/45] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Aug 29 18:57:29 2022] POST /identity/v3/roles => generated 223 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:57:30.086705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 25/46] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:57:29 2022] POST /identity/v3/projects => generated 312 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:57:31.158138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212499d-afe2-43ae-aa48-34a4eeba069a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:31.158660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212499d-afe2-43ae-aa48-34a4eeba069a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:31.158660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212499d-afe2-43ae-aa48-34a4eeba069a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:31.160549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 22/47] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:31.166031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:31.166475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:31.166702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:31.185215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:31.186396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:31.254446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 26/48] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:31.522425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212499d-afe2-43ae-aa48-34a4eeba069a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:31.558504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:31.625312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212499d-afe2-43ae-aa48-34a4eeba069a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:31.765748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:31.766381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:31.766666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:31.766944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:31.777802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:31.782944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:31.783765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:31.784738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 23/49] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:31.790199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:31.998825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30dbef-4d2b-4d62-8456-2a5af4036977 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.015386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:32.032936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.033482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.033759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.034027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.034684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:32.034984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.036400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbe57-b60f-474e-83ab-f3902724a472 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.048752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 27/50] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:32.055068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:32.077602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6baf11c-07a3-4397-ac65-1a5b08b1b1a6 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.077891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6baf11c-07a3-4397-ac65-1a5b08b1b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.078181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6baf11c-07a3-4397-ac65-1a5b08b1b1a6 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.079511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 28/51] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:32.084971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.085249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.085519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.104704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:32.105900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:32.148704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:32.154231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aa3cd-5d00-4922-99ae-c4626df8548a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.171385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 29/52] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:32.187243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.187890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.188174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.188451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.189757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.191559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.193653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 24/53] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:32 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:32.199362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:32.297061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96d709-bde2-4823-b5b7-fa3fa946c846 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.306587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:32.330700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.331333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.331613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.331911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.333016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.334481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.336149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 30/54] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:32.342173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:32.440367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:32.440552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030e998-aa1b-4aca-a071-349d1d731ca4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.475207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.475809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.476118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.476395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.477114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.domain.id=default, user.id=417d5111f71945cab4c66bd4637fb5fb) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:32.477816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.478912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.485326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 25/55] 10.222.0.62 () {56 vars in 1322 bytes} [Mon Aug 29 18:57:32 2022] GET /identity/v3/role_assignments?user.id=417d5111f71945cab4c66bd4637fb5fb&scope.domain.id=default&role.id=ae077013d56245c7869e76158c81fd53 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:32.545848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627b418-8aed-4445-ab81-7fba40ff9d9b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.581610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.582213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.582493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.582768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.583944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.585425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a11ae-5231-42a2-b689-130b60bd658c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.587057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 31/56] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:32.593037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:32.624404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:32.696887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdcb59-95f7-4761-bfb5-f8a5080c45b2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:32.730390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:32.730996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:32.731278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:32.731553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:32.732231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:32.733689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413e81e-4791-4181-b87c-b226406c4e7f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:32.818087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 26/57] 10.222.0.62 () {60 vars in 1140 bytes} [Mon Aug 29 18:57:32 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:34.705778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67615-13a1-4580-a109-a8ed4b18e505 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:34.706284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67615-13a1-4580-a109-a8ed4b18e505 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:34.706371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67615-13a1-4580-a109-a8ed4b18e505 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:34.707689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 32/58] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:34.713056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:34.713449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:34.713705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:34.732563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:34.734062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:34.798291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 27/59] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:35.079078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67615-13a1-4580-a109-a8ed4b18e505 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:35.132350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:35.187676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67615-13a1-4580-a109-a8ed4b18e505 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:35.221624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:35.222249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:35.222526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:35.223669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:35.224797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:35.226240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8c48-972e-427e-9867-3f664baa32ea None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:35.227782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 33/60] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:35.233337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:35.244796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:35.331445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2acb0e-5d7c-4d74-a75b-73189068c705 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:35.365028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:35.365634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:35.365914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:35.366200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:35.366840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:35.368335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40390-4f80-4558-b396-7c54695a798a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:35.480185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 28/61] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:35 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:38.623608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b84a-dca4-4f43-a0c8-86de230a05a8 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.624507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b84a-dca4-4f43-a0c8-86de230a05a8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.624927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b84a-dca4-4f43-a0c8-86de230a05a8 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.626660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 34/62] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:38.632841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01166d-a46e-4274-a1a3-84a18ddaf1b3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.633679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01166d-a46e-4274-a1a3-84a18ddaf1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.634233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01166d-a46e-4274-a1a3-84a18ddaf1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.660859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ac01166d-a46e-4274-a1a3-84a18ddaf1b3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:38.662410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ac01166d-a46e-4274-a1a3-84a18ddaf1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:38.698888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f41-c3c2-4639-92b2-c88a484ea6ee None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.699502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f41-c3c2-4639-92b2-c88a484ea6ee None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.700127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f41-c3c2-4639-92b2-c88a484ea6ee None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.701847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 35/63] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:38.706940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9eb8-9e76-4e44-a294-3cef32e99231 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.707466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9eb8-9e76-4e44-a294-3cef32e99231 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.707969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9eb8-9e76-4e44-a294-3cef32e99231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.741276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-788d9eb8-9e76-4e44-a294-3cef32e99231 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:38.743982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-788d9eb8-9e76-4e44-a294-3cef32e99231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:38.766468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:38.778243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 29/64] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:38.844413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8299c6-6d22-4e9f-99c8-8e1a33243454 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.845007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8299c6-6d22-4e9f-99c8-8e1a33243454 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.845505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8299c6-6d22-4e9f-99c8-8e1a33243454 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.847290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 30/65] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:38.852455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6a38-c584-41ad-8f0c-289f0253458a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.852946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6a38-c584-41ad-8f0c-289f0253458a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.853399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6a38-c584-41ad-8f0c-289f0253458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.873787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 36/66] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:38.885493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-310c6a38-c584-41ad-8f0c-289f0253458a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:38.886721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-310c6a38-c584-41ad-8f0c-289f0253458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:38.993541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 31/67] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:39.060629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6cd9-5a32-4c3c-818a-8feb40b13233 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.060914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6cd9-5a32-4c3c-818a-8feb40b13233 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.061193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6cd9-5a32-4c3c-818a-8feb40b13233 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.062813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 37/68] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:39.067818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.068139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.069042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.078633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946a77-b63e-48c7-8748-00216b455f90 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.078922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946a77-b63e-48c7-8748-00216b455f90 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.079201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946a77-b63e-48c7-8748-00216b455f90 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.080700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 38/69] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:39.086035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.086321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.086597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.106947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:39.108181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:39.115574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:39.116974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:39.160588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.194465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 39/70] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:39.195605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.198437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 32/71] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:39.229434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.301484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29ed09-00c6-4475-8fe8-56f70f34f25f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.301670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.338486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.339497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.340032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.340501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.348826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580b2ac-9e95-4f51-8a43-187d062474e8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.351994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:39.357263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:39.358091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:39.358833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:39.361351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 40/72] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:39.362348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.389795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.390691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.391108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.391520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.395504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.403434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:39.409020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:39.410213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:39.411118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:39.413232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 33/73] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:39.414217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.470012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902c7a-0785-483c-8d3f-e50dd5b7a481 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.507386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.508049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.508337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.508604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.519090 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:39.524153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:39.524962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:39.525698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 41/74] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:39.526326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.533891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bb7cc-c558-4422-905b-ec856e747927 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.550087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.567691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.568450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.569379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.569582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.570247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:39.570549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.571978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.580320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 34/75] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.584960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.619007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.627338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274f19-73db-4cf5-9ad1-e7d074748e08 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.661605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.662429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.662904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.663527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.664276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:39.664582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.665977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.674329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 42/76] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.675124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.682240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908913f-b83a-47ee-9932-5b4200230854 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.716220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.716851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.717132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.717410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.727728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:39.732661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:39.733684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 35/77] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:39.735571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.775839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822774c-4434-4186-a58e-d3c344d4c91f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.776473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.809689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.810324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.810600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.810874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.811531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:39.811835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.813240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.821179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 43/78] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.821959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.836082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28d9d-19e4-477f-ac71-5095ffe2f2cf None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.841084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.873350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.873984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.874271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.874548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.875683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.877188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.878754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 36/79] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.879998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.922176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7aed9-74ac-4010-b389-1e2002682f5b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.954771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.955393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.955671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.955981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.970541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.972027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.973930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 44/80] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.975331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.983085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f977bc-8140-4cde-9d15-60f17335e596 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.991467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.018070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.018706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.018990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.019266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.020440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.021958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.023598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 37/81] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:40.024338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.054864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.078715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41554a-8897-4b8f-b6e9-aa6aa052b5c4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.114080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.114694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.114980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.115377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.116403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:40.116890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.118301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.125035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d67c74-cfa9-4de7-b820-094ef72f96c2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.134044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 45/82] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:40.135066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.160345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.160969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.161247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.161521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.162656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.164157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.170183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 38/83] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:40.170183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.212133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.239606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3b22c-6533-4bbc-90ae-5f1ec0adfced None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.268872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872d556-22cb-4250-99a5-bfb16e121744 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.276467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.277078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.277365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.277637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.278279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.279735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.292431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.303464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.304102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.304379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.304778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.369784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.371281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.394088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 46/84] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:40 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:40.395141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.419330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.491598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 39/85] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:40.493050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.519474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda2bf1-9a4b-40c9-b542-613a12fd5200 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.560770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.560994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.561214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.562507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.620738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca542a-e300-4edc-a75b-5d78d0c33ee8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.633991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.645715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.647344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.660527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.661469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.663872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.664331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.665739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.667287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.669279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 40/86] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:40.670010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.685608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.775733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 47/87] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:40.780967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.792251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898ef5-3aa8-44ee-a917-d0a7a8f86e9c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.829099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.829747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.830032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.830307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.881958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.891037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e192c3-b11c-4f32-8fdc-50f82ee9b80f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.925542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.927155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.929153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.929940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.930314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.930680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.932019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.933637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.935374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 48/88] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:40.936278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:40.941233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.056407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7ff8d-5c04-41d2-9d6b-34e3808ca63d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.076159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 41/89] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:41.077875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.094619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.095311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.095596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.095887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.160234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.186588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.189176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.214944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0265fb-f41c-4dab-bc1a-65f55f6ad2a6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.227102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.258084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.258744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.259284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.259561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.260755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.262499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.265070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 42/90] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.268643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.306773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 49/91] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:41.307390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.369048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.382064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa83bd-3b38-4191-80c9-08c29d5fd7f7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.412062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6571a-8114-4411-a996-aae7509ea2be None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.418459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.419071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.419409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.425809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.425809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:41.425809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.425809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.427961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 43/92] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.436999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.452195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.452857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.453208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.453486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.454702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.456195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.458206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 50/93] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.459615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.512908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.572395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a7b7-e469-41d0-b604-e7868ea50e35 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.590791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aa6c-5af1-40ce-a4ed-0926e8d5e869 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.602725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.611723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.612424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.614668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.614668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] PATH_INFO: `/v3/projects/a37ba409692048f3a143f535aac6a821` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.614668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] RBAC: Authorizing `identity:get_project(project_id=a37ba409692048f3a143f535aac6a821)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.615879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.617858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 51/94] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/projects/a37ba409692048f3a143f535aac6a821 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.618670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.635371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.636088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.636375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.636653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.637620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:41.637994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.639601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.652038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 44/95] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.653084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.737030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760da60-fec4-4873-9bdd-40724a0184f2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.770440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b4bb6-dded-4c5e-8f7c-46b996cc579d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.776868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.777506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.777790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.778071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.778605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.778808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=417d5111f71945cab4c66bd4637fb5fb) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:41.779123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.780616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b11d0-4479-44c7-a2a9-cf71e5024fe7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.790754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 52/96] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/role_assignments?user.id=417d5111f71945cab4c66bd4637fb5fb&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.809975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.810867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.811395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.812419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.813325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=a37ba409692048f3a143f535aac6a821, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:41.813746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.815399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a132d-0cdc-40d8-a16f-1c5d413d443c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.823047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 45/97] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=a37ba409692048f3a143f535aac6a821&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:43.064243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ea25-0413-429b-b194-b568ee128148 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.064778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ea25-0413-429b-b194-b568ee128148 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.064778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ea25-0413-429b-b194-b568ee128148 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.066372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 53/98] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:43.087230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e6bc-543a-4455-93ef-cca481ec95a8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.088226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e6bc-543a-4455-93ef-cca481ec95a8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.088710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e6bc-543a-4455-93ef-cca481ec95a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.115623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3647e6bc-543a-4455-93ef-cca481ec95a8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:43.117418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3647e6bc-543a-4455-93ef-cca481ec95a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:43.206238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 46/99] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:43.359186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e7d3-4b59-45e7-9b82-8be3f7f25baa None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.359479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e7d3-4b59-45e7-9b82-8be3f7f25baa None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.359665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e7d3-4b59-45e7-9b82-8be3f7f25baa None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.361284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 54/100] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:43.366993 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.367280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.367608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.389379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:43.390730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:43.519191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 47/101] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:43.601111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e7d3-4b59-45e7-9b82-8be3f7f25baa None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.690646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.744197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e7d3-4b59-45e7-9b82-8be3f7f25baa None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.789422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.790126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.790414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.790695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.801331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:43.809597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:43.810509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:43.811271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 55/102] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:43.818974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.865213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.953729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cd3ca-f7c6-4506-b718-807029268ecc None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.009475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.010095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.010376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.010653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.011322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:44.011625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.013055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.024049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 48/103] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:44.029916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.036650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.165049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.183754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647225-69e6-45bc-a6b6-cb9a469533a3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.266783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b1ad1-98b5-4bf4-969b-6474f6bf3adf None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.270721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.270721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.270721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.270721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.276633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.279672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.279791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 49/104] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:44.285370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.311332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.339980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49769255-73ef-4774-906e-401e24f80223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.341050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769255-73ef-4774-906e-401e24f80223 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.341533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769255-73ef-4774-906e-401e24f80223 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.342112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769255-73ef-4774-906e-401e24f80223 None admin] PATH_INFO: `/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.358087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49769255-73ef-4774-906e-401e24f80223 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8434e3862b4ff0b235ecb9e9050e3a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.360642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49769255-73ef-4774-906e-401e24f80223 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.362349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 56/105] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a => generated 289 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:44.363540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8255-367c-467b-a7dc-acf51c95eca2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.363826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8255-367c-467b-a7dc-acf51c95eca2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.364126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8255-367c-467b-a7dc-acf51c95eca2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.367427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 57/106] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:44.371259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8255-367c-467b-a7dc-acf51c95eca2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.473664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3a6d-c6e2-47b8-9827-09570d6d505b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.483738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8255-367c-467b-a7dc-acf51c95eca2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.510553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.511489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.512013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.512494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.513340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.515012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae7222-2cbf-4ed3-b1ad-74d280cec9a0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.520803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.521526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.521809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.522095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.578000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 50/107] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:57:44 2022] POST /identity/v3/projects => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:57:44.579702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c258-bfc5-4ad0-b177-cf4502716d3e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.580021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c258-bfc5-4ad0-b177-cf4502716d3e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.580302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c258-bfc5-4ad0-b177-cf4502716d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.586411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.606838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a698c258-bfc5-4ad0-b177-cf4502716d3e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:44.608408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a698c258-bfc5-4ad0-b177-cf4502716d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:44.618732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.622358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.623953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8fb90-6a65-4a82-823b-e14f3dbe8369 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.692638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 51/108] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:44.694176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e2c1a-e08f-433e-b5f0-d9fec764d45e None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.694619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e2c1a-e08f-433e-b5f0-d9fec764d45e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.694903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e2c1a-e08f-433e-b5f0-d9fec764d45e None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.696453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 52/109] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:44.702279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676e4-545d-4bc2-91ad-0b13c3780aee None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.702575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676e4-545d-4bc2-91ad-0b13c3780aee None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.702805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676e4-545d-4bc2-91ad-0b13c3780aee None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.704529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 53/110] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:44.706218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.706504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.706779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.728279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:44.729617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:44.747365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 58/111] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:44.752897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fe57-2730-4316-9f52-d7675d6fa7b5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.753198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fe57-2730-4316-9f52-d7675d6fa7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.753482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fe57-2730-4316-9f52-d7675d6fa7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.774067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5a11fe57-2730-4316-9f52-d7675d6fa7b5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:44.775328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5a11fe57-2730-4316-9f52-d7675d6fa7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:44.804656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 54/112] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:44.812905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.850353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 59/113] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:44.860361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47ccc-5d90-46bf-b4f0-e5ab3ec22905 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.860652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47ccc-5d90-46bf-b4f0-e5ab3ec22905 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.860932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47ccc-5d90-46bf-b4f0-e5ab3ec22905 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.862255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 60/114] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:44.870127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.870415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.870700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.889934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:44.891128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:44.894140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.956928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.965150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 61/115] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:45.010274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.015948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c862-342f-487a-8c1c-071e2eed0fe6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.083694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.084893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.085171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.085431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.088230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.089698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.092852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 55/116] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:45.097475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.193762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b21594-d1c2-4108-83f7-0585021995ba None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.247115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.247757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.248068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.248347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.257902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.262554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:45.268564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.271982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:45.272750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:45.273473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 62/117] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:45.274067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.288695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fe2a-108c-495b-82d4-cd5f48d36b22 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.325047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.325771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.326103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.326448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.327180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=ea8434e3862b4ff0b235ecb9e9050e3a) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.327482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.328974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.336938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 56/118] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/role_assignments?user.id=ea8434e3862b4ff0b235ecb9e9050e3a&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:45.336938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.344907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.412104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0915-6b13-44ae-a399-0318c23fff96 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.470118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.470830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.471176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.471515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.472610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96f32-277e-43c8-bc8b-1a810a5f2bf8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.487783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:45.494937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.495848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 63/119] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:45.500583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.523130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.524173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.524664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.525128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.536061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:45.541960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.542791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:45.543678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:45.545670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 57/120] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:45.546660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.577927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.613953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.637545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c685-a52b-45cc-99d3-09064d554dd1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.654284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242d09-84c6-4497-87e3-87dab5ea7307 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.684223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.684849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.685557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.685859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.686671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.687117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.688879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.694948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.695904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.696391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.696872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.699680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 64/121] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:45.700998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.709197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:45.714912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.715739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:45.716527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:45.718777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 58/122] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:45.723881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.831874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220a37-5e1e-40fe-84fd-201e0b8a3b30 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.846420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c389-f86f-4c81-9ba3-5bbd0662d1a4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.859372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.874106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.874740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.875080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.875425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.876247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.876621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.878109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.884620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.888206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 65/123] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:45.889121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.889835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.890827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.891106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.891379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.892075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.892949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.894908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.906912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 59/124] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:45.907796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.002676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f53fc-0898-41df-9224-cc8463825f7b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.026754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97280f0-4321-4547-a5be-53081b455a06 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.047699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.048544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.048916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.049707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.051235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.053940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.060297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 66/125] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.060297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.069852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.070802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.071276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.071840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.072751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:46.073247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.075148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.085371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 60/126] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.086800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.125214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.154458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.181106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a5ca-1813-406c-a5be-45707ed87fe8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.209924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c0517-16ba-4839-aeca-a85d756ae14b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.219968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.221029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.221525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.222049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.223701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.226093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.228434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 67/127] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.229480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.244874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.245517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.245797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.246071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.247285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.248674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.250396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 61/128] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.254832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.349670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf97e40-5527-41a1-bfc7-951896cd6b88 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.354013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5103ab-414d-4842-9ce5-f66424f43bae None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.371049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.373704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.389301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.389850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.390140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.390392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.391599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.393159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.395094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 62/129] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.395606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.395868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.397276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.397761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.398283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.401234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.403185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.405631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 68/130] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.411543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.562254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7a59-1735-45a6-8244-2ea9da2c0987 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.565418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eb1e1-1996-4ce2-922e-502adccbf52a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.602280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.603318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.603794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.604305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.604876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.605611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.605889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.606163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] PATH_INFO: `/v3/projects/a37ba409692048f3a143f535aac6a821` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.606524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.607274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] RBAC: Authorizing `identity:get_project(project_id=a37ba409692048f3a143f535aac6a821)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.608439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.608747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.610377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 69/131] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/projects/a37ba409692048f3a143f535aac6a821 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.615381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.615734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 63/132] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.616859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.671971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.674354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.718898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc44da-ce8d-4816-b07d-dca0702055be None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.725841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7093034-0405-40de-bf93-fa5707ae9cca None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.754508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.755197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.755963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.756256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.757386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.758882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.765271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 70/133] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:46 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:46.766193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.768849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.769746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.770247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.770725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.773762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, user_id=9618dcb87a9040ebb47c80ebac85c6cf, role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.775540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:46.841642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 64/134] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:46 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:46.851895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:46.875094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d60b88-4954-4c9b-a218-36ca357db073 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:46.901380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:46.914131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:46.915039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:46.915483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:46.915912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] PATH_INFO: `/v3/projects/a37ba409692048f3a143f535aac6a821/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:46.922927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] RBAC: Authorizing `identity:create_grant(project_id=a37ba409692048f3a143f535aac6a821, user_id=9618dcb87a9040ebb47c80ebac85c6cf, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:46.922927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:47.013836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 71/135] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:46 2022] PUT /identity/v3/projects/a37ba409692048f3a143f535aac6a821/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:47.015526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:47.059237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e650-3d14-49bd-8582-f20888effcd7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:47.075548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:47.107607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:47.108634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.108634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.108832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.111693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, user_id=9618dcb87a9040ebb47c80ebac85c6cf, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:47.113241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f3231-fa64-4cb1-b777-7312882877ec None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:47.160076 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27cb6e-f05b-481c-9e52-2f3d675eab28 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:47.193331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 65/136] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:46 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/9618dcb87a9040ebb47c80ebac85c6cf/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:47.197855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996013-52f0-4eb4-9d03-e43655cb2fd5 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.198131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996013-52f0-4eb4-9d03-e43655cb2fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.198410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996013-52f0-4eb4-9d03-e43655cb2fd5 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.199971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 66/137] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:47.207225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:47.208254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.209280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.209732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/417d5111f71945cab4c66bd4637fb5fb/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.213267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, user_id=417d5111f71945cab4c66bd4637fb5fb, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:47.215455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82fbf-cdcc-418c-b220-b2f12351a373 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.215739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82fbf-cdcc-418c-b220-b2f12351a373 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.216079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82fbf-cdcc-418c-b220-b2f12351a373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.219887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73544c12-07ff-4850-acce-24b76a405dd8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:47.240729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4eb82fbf-cdcc-418c-b220-b2f12351a373 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:47.242040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4eb82fbf-cdcc-418c-b220-b2f12351a373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:47.297556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 72/138] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:47 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/417d5111f71945cab4c66bd4637fb5fb/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:47.326142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 67/139] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:47.463663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2026c5c-de3b-477c-9818-05d252f87105 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.464214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2026c5c-de3b-477c-9818-05d252f87105 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.464214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2026c5c-de3b-477c-9818-05d252f87105 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.465889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 73/140] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:47.472019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.472494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.472593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.497229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:47.498515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:47.574923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 68/141] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:47.732402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2026c5c-de3b-477c-9818-05d252f87105 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:47.797131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:47.869759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2026c5c-de3b-477c-9818-05d252f87105 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:47.913331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:47.914361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.914840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.915326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.916674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:47.918361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:47.920232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 74/142] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:47.928940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:47.980850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:48.048268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.066631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e5294-7338-4d3f-95b9-9ca26cc2d0c3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.106696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.107369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.107650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.107945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.108632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.110099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.204797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2638-f80b-49aa-bd41-6decc306895f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.218725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 69/143] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:47 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:48.219241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:48.251917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.252483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.252756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.253023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.265610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:48.270934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:48.272472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:48.273223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 75/144] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:48.280037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.382235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:48.399417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93471352-01e9-430d-9bca-7d1ce9f6a97c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.452282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.452968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.453253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.453526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.454197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:48.454499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.455970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.464892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 70/145] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:48.480762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.611821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca8e2-a99b-4c76-9f38-9816418ff092 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.656349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.657310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.657828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.658291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] PATH_INFO: `/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.659622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] RBAC: Authorizing `identity:get_user(user_id=ea8434e3862b4ff0b235ecb9e9050e3a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.663835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 76/146] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:48.671371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.821830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad53ae-1040-4b5b-90bc-908fe7439ea1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.870709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.871285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.871605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.871914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.873203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.874864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845d0a3-a470-4b21-88d5-2f45a5b41dcd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.876723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 71/147] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:48.883166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.948323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:49.039508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9105-53a7-47de-a061-ab3836fd6f0c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.083331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.084075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.084365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.084626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/ea8434e3862b4ff0b235ecb9e9050e3a/roles/5f31d1e396bf46d7a0ba15862b3d18ba` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.089141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, user_id=ea8434e3862b4ff0b235ecb9e9050e3a, role_id=5f31d1e396bf46d7a0ba15862b3d18ba)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.090596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc236bf-5bc0-4d60-8490-3f503143a619 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.206422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 77/148] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:48 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/users/ea8434e3862b4ff0b235ecb9e9050e3a/roles/5f31d1e396bf46d7a0ba15862b3d18ba => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:49.659496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec919e7-5a5d-4365-958e-19d7527751b5 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.660188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec919e7-5a5d-4365-958e-19d7527751b5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.660673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec919e7-5a5d-4365-958e-19d7527751b5 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.662270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 72/149] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:49.669361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5daa45-15e3-4dcd-9571-f034e96ed209 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.670366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5daa45-15e3-4dcd-9571-f034e96ed209 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.670860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5daa45-15e3-4dcd-9571-f034e96ed209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.696872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9c5daa45-15e3-4dcd-9571-f034e96ed209 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:49.698420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9c5daa45-15e3-4dcd-9571-f034e96ed209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:49.789191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 78/150] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:50.024897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580010-5a31-460d-9fc1-2dc259860d9f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.025848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580010-5a31-460d-9fc1-2dc259860d9f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.026337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580010-5a31-460d-9fc1-2dc259860d9f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.027917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 73/151] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:50.033645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc6b50-e4b5-4796-8748-0c3011d75e51 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.034205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc6b50-e4b5-4796-8748-0c3011d75e51 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.034699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc6b50-e4b5-4796-8748-0c3011d75e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.062763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c7cc6b50-e4b5-4796-8748-0c3011d75e51 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:50.064307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c7cc6b50-e4b5-4796-8748-0c3011d75e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:50.095082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.155596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 79/152] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:50.215404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced50a0-f494-47d2-8062-e0bda020bbb1 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.216065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced50a0-f494-47d2-8062-e0bda020bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.216539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced50a0-f494-47d2-8062-e0bda020bbb1 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.218086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 74/153] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:50.224020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85c4-20bf-4716-b47b-a01b161b7f62 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.224573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85c4-20bf-4716-b47b-a01b161b7f62 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.225073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85c4-20bf-4716-b47b-a01b161b7f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.249234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0bee85c4-20bf-4716-b47b-a01b161b7f62 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:50.256535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0bee85c4-20bf-4716-b47b-a01b161b7f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:50.304829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced50a0-f494-47d2-8062-e0bda020bbb1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.344994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.360680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 80/154] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:50.461336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced50a0-f494-47d2-8062-e0bda020bbb1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:50.490439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefe70-b829-4714-bb9c-544600b95090 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.490732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefe70-b829-4714-bb9c-544600b95090 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.490986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefe70-b829-4714-bb9c-544600b95090 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.492521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 81/155] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:50.502794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.503023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.503270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.514477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:50.515113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.515431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.515658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.534983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:50.536852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:50.538069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:50.540521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:50.541365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:50.542325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 75/156] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:57:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:50.550568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.690715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.721365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 82/157] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:50.734774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f89dbb-ef6c-4c9d-bc2a-04f663ff4f69 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:50.800712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:50.801735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.802750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.803235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.804253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:50.804832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:50.806430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:50.815384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 76/158] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:50.822030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.853444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.899180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.966702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604cdde-65d9-4ab5-a2e1-16d6a3032bbd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.024585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.025235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.025512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.025795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.026973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:51.028481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:51.030253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 83/159] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:50 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:51.036926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.113102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0776c1-1f07-4b69-bcc6-23583f840d91 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.159699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:51.175068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.175711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.177833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.177833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.179882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8dc66-052d-429f-875d-6dbcf4d53460 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.193647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:51.202045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:51.207462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:51.208434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:51.209224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 77/160] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:51.216483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.240576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.241230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.241487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.241834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.313343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:51.314825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:51.408065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:51.409759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b1f12-626a-4982-b9a5-c905f82bb579 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.424212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 84/161] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:57:51.425068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.476375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.477092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.477401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.477747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.478413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:51.478741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:51.480196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:51.499644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 78/162] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:51.500630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.529270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe44c9-be0a-49a2-9dfc-456743997c93 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.571205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.572194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.572477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.572750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.593555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:51.603330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:51.604200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:51.604988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 85/163] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:51.605616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.628238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:51.698663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:51.702259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f0e7-5a57-44f4-8251-428bef924128 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.775952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.777062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.777543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.778006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.799514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:51.806893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:51.808056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:51.808837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:51.811885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3bd45-e37a-4fd5-ac01-ed44e069afba None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:51.820810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 79/164] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:51.821710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:51.870023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:51.870664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:51.870942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:51.871217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:51.872981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:51.874457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:51.876313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 86/165] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:51.876925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.004253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd474cd7-ad82-4aab-abc1-6b2575b233a3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.020219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.041678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c34f01-bf5a-42c2-bce9-1f57dd0cd489 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.044487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.045405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.045873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.046343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.047213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:52.047715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.049371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.061541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 87/166] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.062656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.080967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.104370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.105021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.105509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.105613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.106731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.108259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.110009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 80/167] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:51 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.110999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.189716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334419b8-c6c0-4cab-aa7f-fc3564e7421a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.228997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.229971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.230536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.230996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.231882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:52.232489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.234124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.243778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 88/168] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.249238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.299063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.371599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203334a-e36d-42f3-b235-0b1ff3688f6f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.394193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740be8d-990f-41f9-9944-72f198c2d6fa None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.414140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.415326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.415846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.416347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] PATH_INFO: `/v3/users/417d5111f71945cab4c66bd4637fb5fb` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.417747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] RBAC: Authorizing `identity:get_user(user_id=417d5111f71945cab4c66bd4637fb5fb)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.419485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.421532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 89/169] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/users/417d5111f71945cab4c66bd4637fb5fb => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.422633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.460147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.460905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.461148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.461481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.462441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:52.462668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.464180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.471210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 81/170] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 608 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.476373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.550162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eeda3-19a9-44c4-b778-4ad26709141d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.588232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.592315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.594001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.594482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.594975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.620264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.650233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeceeb4-3ea4-4dac-9ae8-78cd1f86261b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.696282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.697826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206fa4-7e48-4e4f-ba8b-6125e4c87e02 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.699474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.700214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.700479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.700748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] PATH_INFO: `/v3/users/9618dcb87a9040ebb47c80ebac85c6cf` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.701959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] RBAC: Authorizing `identity:get_user(user_id=9618dcb87a9040ebb47c80ebac85c6cf)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.703417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.710437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 82/171] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/users/9618dcb87a9040ebb47c80ebac85c6cf => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:52.711639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.825480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 90/172] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:52.826713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56acbe-cceb-48c3-b7c8-e1cd779c3965 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.826999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56acbe-cceb-48c3-b7c8-e1cd779c3965 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.827274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56acbe-cceb-48c3-b7c8-e1cd779c3965 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.828745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 91/173] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:52.829619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05adc64-6866-4b77-b8f0-ed3f78718154 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.829896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05adc64-6866-4b77-b8f0-ed3f78718154 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.830387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05adc64-6866-4b77-b8f0-ed3f78718154 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.835297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 92/174] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:52.835297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05adc64-6866-4b77-b8f0-ed3f78718154 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:52.846506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.852637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76d0f-e7fa-4cab-82ee-0254e01d2661 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.884253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:52.891450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.892448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.893196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.893814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.950948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05adc64-6866-4b77-b8f0-ed3f78718154 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:52.976792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:52.978839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:52.989401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:52.990051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.990369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.990825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.061922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.063431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8044cf-3d2f-4cab-9f45-36898e5ba7ee None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.071391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.092342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 83/175] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:53.097469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.122104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.169089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 93/176] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:53.170206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57d96-3f47-47f3-be2c-819aac218578 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.170493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57d96-3f47-47f3-be2c-819aac218578 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.170768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57d96-3f47-47f3-be2c-819aac218578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.188559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6ff57d96-3f47-47f3-be2c-819aac218578 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:53.189755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6ff57d96-3f47-47f3-be2c-819aac218578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:53.204257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ecfb0-17cc-4027-b663-2ad2b4c4958c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.239936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.240570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.240843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.241104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] PATH_INFO: `/v3/projects/a37ba409692048f3a143f535aac6a821` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.242229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] RBAC: Authorizing `identity:get_project(project_id=a37ba409692048f3a143f535aac6a821)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.243681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ecbe4-e3d5-4846-840e-92f1d1c49a30 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.248154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 84/177] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/projects/a37ba409692048f3a143f535aac6a821 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.249232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.249528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.249812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.263173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 94/178] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:53.264732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc803f-86ff-4454-96dc-145e757c073e None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.265016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc803f-86ff-4454-96dc-145e757c073e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.265290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc803f-86ff-4454-96dc-145e757c073e None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.266542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 95/179] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:53.267245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc803f-86ff-4454-96dc-145e757c073e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.285468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:53.287270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:53.307262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.377304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc803f-86ff-4454-96dc-145e757c073e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.380996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 85/180] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:53.384738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.408645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.464798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.465270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.465596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.465885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.467067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.468551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.470631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 96/181] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.471449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.507658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541f5d0-e036-4296-9648-3c26e52bf87d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.554551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.555846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.556353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.556833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.558282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.560091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7fea6-5b6e-48b2-8b5f-1ac72c8c6d82 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.562842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 86/182] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.568800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.569358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.569832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.590758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:53.592295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:53.673694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 87/183] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:53.675077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.682210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff68d-3f88-4af7-ab1f-4ceed025ac35 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.685927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.713103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.718850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.720010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.720010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.720010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.721576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=a37ba409692048f3a143f535aac6a821, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:53.721576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.722965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.740450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 97/184] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=a37ba409692048f3a143f535aac6a821&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.741524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.878866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d523-9b91-45a0-9723-e02927180a42 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.929083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.929680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.929958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.930224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.931002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=417d5111f71945cab4c66bd4637fb5fb) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:53.931307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.932883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.941196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 88/185] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/role_assignments?user.id=417d5111f71945cab4c66bd4637fb5fb&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.941978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.967818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10cdd-4976-4e6e-bc53-e8badf02fc8b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.035827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.038516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.038516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.038516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.056380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-ee49f969-2947-43f4-ac8b-72b9a68e26ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:54.061788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:54.062707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:54.063492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 98/186] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:54.064191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9a35-76c4-472f-acaa-b9978d8bcb37 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.064795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9a35-76c4-472f-acaa-b9978d8bcb37 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.064795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9a35-76c4-472f-acaa-b9978d8bcb37 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.071029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 99/187] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:54.071029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9a35-76c4-472f-acaa-b9978d8bcb37 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.090888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.105407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.112176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf189eab-19bc-47d9-9234-ca64944b22bd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.153525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.153750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.153963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.154230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.154953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=9618dcb87a9040ebb47c80ebac85c6cf) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.155254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.157627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.165127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 89/188] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/role_assignments?user.id=9618dcb87a9040ebb47c80ebac85c6cf&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.165956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.200142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9a35-76c4-472f-acaa-b9978d8bcb37 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.243294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.243982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.244263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.244849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.258188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-a1d34fc8-1752-43c8-8878-a75deb25150b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:54.263303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:54.264244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:57:54.265028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 100/189] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:54.266579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.266579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.266579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.286109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:54.287876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:54.295871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176aab0-ddc4-480c-9a91-09ba6bb2233f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.337294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.337929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.338205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.338493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.353044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:54.358160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:54.359050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:54.359818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 90/190] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:54.361320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.370926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.377335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 101/191] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:54.384785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.406223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.477106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1765ba-1e2b-4ac9-8bc2-6d127c2f98bd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.492788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d563f0-84e8-43e7-a181-aa05fb636b90 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.515011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.515737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.516047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.516432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.517090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.517393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.518882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.527435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.528060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.528334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.528743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 91/192] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.529264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.532950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.533677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.533986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.535362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.549341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 102/193] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.549341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.609651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.632742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.646485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc099b-b07c-4eaf-8f2e-a45b23eb8195 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.652240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc6b2-9499-4a01-a54b-a74c179a54c7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.682519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.683291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.683633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.684113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.685016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.685390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.687030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.688725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.689343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.689618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.689884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.697169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 92/194] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.697985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.703882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.705329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.707142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 103/195] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.708008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.800675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6841b6-963b-4859-a35c-91fbac67ef11 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.807937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a8e58-a4b1-4184-8b2a-56feb3bd9759 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.827813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.833552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.834174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.834445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.834714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] PATH_INFO: `/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.835900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] RBAC: Authorizing `identity:get_user(user_id=ea8434e3862b4ff0b235ecb9e9050e3a)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.837292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.842534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 93/196] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/users/ea8434e3862b4ff0b235ecb9e9050e3a => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.842534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.844326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.845039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.845319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.845591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.846717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.848181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.850494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 104/197] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.851254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.860272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.942836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0bcec-e8e0-48e5-b22a-69e7579fb45d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.950604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59e71-000a-40e2-a742-a2af88ed608e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.976987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.977622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.977900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.978172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.979297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.980793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.982356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 94/198] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.983120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.990796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.990796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.990796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.990796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.048379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.060605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.062014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.070046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.083396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa7d61-38e1-4a91-aac2-27d6f96a3f3f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.116988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.117595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.117872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.118141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.160811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 105/199] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:55.161695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.187837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.189344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.252253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.260175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d015-17ef-4a54-80e6-29b34f4b1b08 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.267510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.286698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 95/200] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:55.287523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.298492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.299131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.299410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.299684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.300368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.301815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.389358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f85ea-1fda-489f-ae3c-2e8f46bf71fd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.411675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 106/201] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:55 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:55.413299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.430631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.431266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.431530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.431800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.488409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.507331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.508850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.526474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b8ee-4cf9-4fc5-ba8b-91e862e1dfe7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.569126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.569824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.570103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.570403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.571776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.573334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.575303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 107/202] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:55.576788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.615843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.631819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 96/203] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:55.632490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.705905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147dbd-e57c-4891-9043-9f15cf10d3d1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.715386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.747230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.748034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.748271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.748544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.751332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.754269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.760830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132b6a8-92c1-4e3c-bb33-adb9f44d27ec None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.761380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 108/204] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:55.761380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.802692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.803462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.803845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.804287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.805127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=a729c8d183dc46c68984ba990ceb7036) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:55.805508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.807178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.814564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 97/205] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/role_assignments?user.id=a729c8d183dc46c68984ba990ceb7036&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:55.815699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.947595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191cf52-2329-41e8-a408-c7bba0542ffd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.952852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.967266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03f4e-e7a7-4cfc-9d31-e08bb332a0fa None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:56.002057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:56.010966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa, user.id=ea8434e3862b4ff0b235ecb9e9050e3a) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:56.016350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:56.022475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:56.023125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:56.023406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:56.023688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:56.024088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 98/206] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/role_assignments?user.id=ea8434e3862b4ff0b235ecb9e9050e3a&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:56.031558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:56.031558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525da632-e873-4ee3-8051-52fa653e4191 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:56.031558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 109/207] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:56.039187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:56.195565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b21-46f8-4d56-babb-bdec2fc720e4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:56.246115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:56.247152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:56.247664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:56.248243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:56.249211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=a729c8d183dc46c68984ba990ceb7036) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:56.249736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:56.251443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac08b-e385-4ee4-8079-2e138816a280 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:56.262311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 99/208] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:56 2022] GET /identity/v3/role_assignments?user.id=a729c8d183dc46c68984ba990ceb7036&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:57.633660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c45f5-3cd0-4680-ba41-7e069d2ae721 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.634194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c45f5-3cd0-4680-ba41-7e069d2ae721 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.634194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c45f5-3cd0-4680-ba41-7e069d2ae721 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.635544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 110/209] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:57.641053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49411-18f2-4f12-9ef3-3c45737c6aba None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.641939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49411-18f2-4f12-9ef3-3c45737c6aba None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.642428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49411-18f2-4f12-9ef3-3c45737c6aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.661325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-32a49411-18f2-4f12-9ef3-3c45737c6aba None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:57.662951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-32a49411-18f2-4f12-9ef3-3c45737c6aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:57.703455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91741c46-48ac-4652-aea0-f4026b0d7797 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.704141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91741c46-48ac-4652-aea0-f4026b0d7797 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.704621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91741c46-48ac-4652-aea0-f4026b0d7797 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.706401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 111/210] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:57.712125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.712695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.713178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.719047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:57.734529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:57.735566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 100/211] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:57.736515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:57.792637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f12b-413f-48b2-b47b-7814934ce30c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.792971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f12b-413f-48b2-b47b-7814934ce30c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.793184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f12b-413f-48b2-b47b-7814934ce30c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.794533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 101/212] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:57.799497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:57.799777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:57.800087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:57.812249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 112/213] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:57.819411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:57.820625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:57.891200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 102/214] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:57:58.007720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.099774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.111505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.118448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b61dcd-4a43-4db0-a2f9-41fc074d2e36 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.161309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.162001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.162279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.162831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.163984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:58.165853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:58.167955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 113/215] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:58.172993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.222117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bbc3a-4b13-4057-af0c-748c93cff602 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.241166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.264888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.265521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.265799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.266072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.280043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:58.280198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fc3d6-5f53-4b4e-9ba2-934a9e2a38a9 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:58.285608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:58.287383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:58.288260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 103/216] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:58.288962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.316132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.316741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.317020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.317289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.318518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:58.319973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564a524-21a4-46f6-9f86-7cf29cbb94c0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:58.387831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0824c5-2049-4171-aff2-83e628c4fa71 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.405091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 114/217] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:58 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:57:58.414151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fe177-3c95-4e3b-9d80-027cf77513db None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.414151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fe177-3c95-4e3b-9d80-027cf77513db None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.414151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fe177-3c95-4e3b-9d80-027cf77513db None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.414151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 115/218] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:57:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:58.414151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fe177-3c95-4e3b-9d80-027cf77513db None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.432078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.432731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.433102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.433299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.446082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.446591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-1caa0ed8-505b-4fa0-8141-1617c435b4a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:58.451953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:58.452902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:58.454421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 104/219] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:58.455120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.455120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.455120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.475327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:58.478835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:58.513568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.543964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fe177-3c95-4e3b-9d80-027cf77513db None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.560183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 105/220] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:57:58.561605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.583722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.584461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.584744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.585017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.585684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:58.585983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:58.587374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:58.599105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 116/221] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:58.605843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.789496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8176d2-d79b-4c7b-a64f-9700be70d9db None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.827291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.831716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.832418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.832695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.832999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.834347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:58.834653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:58.836787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:58.846979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 106/222] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:58.847955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.856033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6093-e326-46e7-a510-792715bc3a47 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.882493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.909140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.910089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.910371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.910837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] PATH_INFO: `/v3/users/0ccb0908e8ae45e084f562629dbd021d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.931705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] RBAC: Authorizing `identity:get_user(user_id=0ccb0908e8ae45e084f562629dbd021d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:58.933520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:58.936725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 117/223] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/users/0ccb0908e8ae45e084f562629dbd021d => generated 297 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:58.937532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.978324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d7048-80a4-4b38-b6e8-320bdc8e6126 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.021956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.022915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.023366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.023825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.035435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:59.041512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:59.042371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:57:59.043178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:57:59.045656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 107/224] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:59.046724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.070248 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70e7f-236c-4891-b3c8-8b5ad52d5640 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.119070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.119725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.120039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.120330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.121722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.123436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.125690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 118/225] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.126985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.144210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.152158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.163898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4995d-6c4a-4ec3-ab0c-5bbf864d7481 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.209760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.210363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.210646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.210916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.260226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc38766-c5db-46d4-ad5f-76c971dc01ee None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.287081 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.288917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.307387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.308605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.309043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.309473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.310280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:59.311241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.313075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.325404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 119/226] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.326610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.403235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.416518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 108/227] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:57:59.417660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.428400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.458578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d022e-55a5-4cec-a1fc-c25bf8cf7bf2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.505743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.507968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.508446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.508902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.510303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.512582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.514989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 120/228] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.520112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.546784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd9fff-688d-4ca7-801b-acd50845d72e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.590477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.591033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.591311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.591584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.593323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.595106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.597299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 109/229] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.603224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.652000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd1b72-d980-42ae-8ecd-6e659e43da9b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.685154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.688946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.689728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.690020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.690292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.691761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.694145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.702831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 121/230] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.702831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.713844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.750102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f85c90-00eb-49ce-83d0-9d546dc904d0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.801873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.802566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.802991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.803345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/a729c8d183dc46c68984ba990ceb7036/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.806477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, user_id=a729c8d183dc46c68984ba990ceb7036, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.808466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.829325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254361-9014-4030-8c55-1b1cdf2926ed None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.865748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.866459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.866828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.867109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.868260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.869712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.871841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 122/231] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.872746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.887984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 110/232] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:59 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/a729c8d183dc46c68984ba990ceb7036/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:59.889821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:00.016421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:00.038708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601ac6-9cbd-40ef-a7e9-9c1545fd67b2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:00.062495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:00.085387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:00.086704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.087185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.087662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/a729c8d183dc46c68984ba990ceb7036/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.089483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d24cdf-d896-4dac-88d3-a575907304ec None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:00.090359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, user_id=a729c8d183dc46c68984ba990ceb7036, role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:00.092400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3deee0-4e34-4bcd-b66f-d4a2e04f2f2c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:00.136272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:00.136915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.137206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.137583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.138319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=0ccb0908e8ae45e084f562629dbd021d) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:00.138740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:00.140284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e57d8-8ad5-4ac0-aecb-66fa90050a81 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:00.147725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 123/233] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/role_assignments?user.id=0ccb0908e8ae45e084f562629dbd021d&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:00.240820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 111/234] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:59 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/a729c8d183dc46c68984ba990ceb7036/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:00.571124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627b45-71a9-40be-9ac2-4c0ddfc8cb5a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.571496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627b45-71a9-40be-9ac2-4c0ddfc8cb5a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.571672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627b45-71a9-40be-9ac2-4c0ddfc8cb5a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.573311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 124/235] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:00.578539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649103-0911-4892-b235-cc51e1420c97 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.578823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649103-0911-4892-b235-cc51e1420c97 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.579099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649103-0911-4892-b235-cc51e1420c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.597764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-34649103-0911-4892-b235-cc51e1420c97 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:00.599743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-34649103-0911-4892-b235-cc51e1420c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:00.667423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 112/236] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:00.807022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab363d-6837-45e5-97b8-373274364a73 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.807401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab363d-6837-45e5-97b8-373274364a73 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.807572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab363d-6837-45e5-97b8-373274364a73 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.809194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 125/237] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:00.814328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.814779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.814856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.844955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:00.846093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:00.939488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 113/238] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:00.942614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab363d-6837-45e5-97b8-373274364a73 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.079449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab363d-6837-45e5-97b8-373274364a73 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.118844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.119803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.120327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.120790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.131877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:01.136328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:01.141449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:01.149401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:01.150195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 126/239] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:01.150867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.220426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:01.301876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e26a2-e522-4934-abbd-9de0bb5687f6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.330465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.359796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.360429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.360705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.360974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.361636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:01.361936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.363332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.383523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 114/240] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:01.388906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.436333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26fbb9-1106-493f-b5d7-d82a98065c50 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.470919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.471558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.471836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.472314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.473443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.475181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.476807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 127/241] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:01.482147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.581904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:01.622962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb3fff-b6e7-4428-a354-53dfd4176f14 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.643244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:01.646241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ba396-97b1-49b8-8ccb-eb9099111556 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.672364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.672977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.673252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.673521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.674170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.675612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.702734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.703413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.703695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.704002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] PATH_INFO: `/v3/users/6fcee8f4bb2b47b392784b094610a274` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.717712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] RBAC: Authorizing `identity:get_user(user_id=6fcee8f4bb2b47b392784b094610a274)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.719280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.721209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 115/242] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/users/6fcee8f4bb2b47b392784b094610a274 => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:01.727637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.780629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 128/243] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:01 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:01.876260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271346-e010-4971-bf96-f4a88317d6bd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.931100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.931732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.932034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.932306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.968756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:02.020154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:02.022366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:02.132200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 116/244] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:02.137734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:02.267916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fce1-41b2-48d8-af82-9b25f3c05ffd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:02.295152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:02.312810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:02.313460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.313741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.314014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.315153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:02.316656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77239e-a2ce-4799-b48c-84c1fd22ed3c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:02.319456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 129/245] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:02 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:02.325603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:02.407561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:02.461308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadb4e-eec2-4a33-a828-e12c6cc1233d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:02.498235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:02.499169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.499449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.499930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.500673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=6fcee8f4bb2b47b392784b094610a274) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:02.500984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:02.502439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1f0ef-2c85-4a44-b7b0-a00b68ae94b5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:02.509175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 117/246] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:02 2022] GET /identity/v3/role_assignments?user.id=6fcee8f4bb2b47b392784b094610a274&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:02.539490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e25f-e773-4280-86ec-f3358d22a2fe None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.539797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e25f-e773-4280-86ec-f3358d22a2fe None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.540033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e25f-e773-4280-86ec-f3358d22a2fe None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.541405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 130/247] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:02.547481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bf373-9c33-426f-bd24-88d54a2a6180 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.547773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bf373-9c33-426f-bd24-88d54a2a6180 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.548072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bf373-9c33-426f-bd24-88d54a2a6180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.574328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ca8bf373-9c33-426f-bd24-88d54a2a6180 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:02.575631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ca8bf373-9c33-426f-bd24-88d54a2a6180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:02.590654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f429f8-34a6-41c9-b05b-8345194a7ade None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.591278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f429f8-34a6-41c9-b05b-8345194a7ade None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.591752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f429f8-34a6-41c9-b05b-8345194a7ade None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.593559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 131/248] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:02.600237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.600824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.601160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.636339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:02.637671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:02.661266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 118/249] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:02.662640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81533fd-ec40-45c0-b8b8-65b19475280a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.662926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81533fd-ec40-45c0-b8b8-65b19475280a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.663195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81533fd-ec40-45c0-b8b8-65b19475280a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.669120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 119/250] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:02.675508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:02.675796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:02.676093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:02.705559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:02.706763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:02.736638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 132/251] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:02.767922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:02.798491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 120/252] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:03.017231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.024942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.142596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073d4c-3233-45aa-9ded-6c8ee2fc7283 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.196924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.197569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.197839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.198107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.210153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0f453-8847-443c-a9e0-f7852a4c4cd4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.227499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:03.237689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:03.238622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:03.239463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 121/253] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:03.250593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:03.264524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.281127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.281779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.282053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.282321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.324480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:03.329421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:03.330273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:03.331056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 133/254] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:03.340215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.340552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:03.442009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab925a1-def8-480d-9f3c-ec3cb152e146 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.511758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.512760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.513039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.513310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.513975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:03.514274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:03.516631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:03.531289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a084d-9f1f-4ced-bf8b-afe5cba3f180 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.537125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 122/255] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:03.538000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.625081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.625733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.626016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.626286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.637160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:03.642056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:03.643096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:03.643896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 134/256] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:03.644556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.654783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62d375-ef90-42ef-a1e6-a3dfab3f9415 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.657917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:03.714823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.715324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:03.715435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.715648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.715965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.716815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:03.717123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:03.718597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:03.729320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 123/257] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:03.730157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.837380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5678d0-864d-47da-a324-92b2024d9ed4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.879059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8b46f-9ac2-4164-8cf2-a23c23e58c57 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:03.902752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.903398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.903672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.903972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.905365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:03.906824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:03.908429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 135/258] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:03.911587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.943732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:03.944388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.944666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.944934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.945592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:03.945890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:03.947278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:03.956556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 124/259] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:03.957305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.993172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.122001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b440c-62a5-4e01-afc1-3f95a2c97670 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.124353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc71a1-4aa9-4b07-8cbe-8aa492b56c54 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.144459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.187359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.188360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.189221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.189502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.190619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.191245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.191520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.191791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] PATH_INFO: `/v3/users/0ccb0908e8ae45e084f562629dbd021d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.194818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] RBAC: Authorizing `identity:get_user(user_id=0ccb0908e8ae45e084f562629dbd021d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.196243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.200664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 136/260] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/users/0ccb0908e8ae45e084f562629dbd021d => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.210100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.322680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.324801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.380382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.400523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f38023-b689-4944-96ff-440b6ec84eaa None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.457598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.458707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.459184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.459650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] PATH_INFO: `/v3/users/a729c8d183dc46c68984ba990ceb7036` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.464501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c8d183dc46c68984ba990ceb7036)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.466230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.470153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 137/261] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/users/a729c8d183dc46c68984ba990ceb7036 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.472522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.530955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 125/262] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:04.532077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.572472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.678096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f572c-d4af-44cb-845d-780caf267a01 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.731763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a2799-0f02-4500-b0b7-6e04319b555b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.751365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.752395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.752875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.753367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.756941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.758631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.763303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 138/263] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.765396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.793139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.809771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.810844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.811996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.813841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.893803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.895391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff653-6c10-4dd7-b548-3c3d064af2dc None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.896006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e813f0-7cae-4cc1-94ca-ddaecbc75fca None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.909979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.931340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.931924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.932287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.932473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.933584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.935014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.941580 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 139/264] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.942853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.006941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 126/265] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:05.008240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-794d-4a26-86ab-3192a7c36ddf None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.008525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-794d-4a26-86ab-3192a7c36ddf None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.008797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-794d-4a26-86ab-3192a7c36ddf None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.010163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 127/266] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:05.011067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3616d7-a0ef-459d-81b1-d51b82799c32 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3616d7-a0ef-459d-81b1-d51b82799c32 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.011734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3616d7-a0ef-459d-81b1-d51b82799c32 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.013018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 128/267] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:05.013652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3616d7-a0ef-459d-81b1-d51b82799c32 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.043806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.055210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35f7f7-9abe-475b-b704-5723ffe8fbf0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.088858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.089511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.089789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.090059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/0ccb0908e8ae45e084f562629dbd021d/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.091993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, user_id=0ccb0908e8ae45e084f562629dbd021d, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.093441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409097e4-e4e4-4c72-bbee-2ea44f5241ad None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.116238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3616d7-a0ef-459d-81b1-d51b82799c32 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.148438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.149025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.149296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.149560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.150266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=a729c8d183dc46c68984ba990ceb7036) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:05.150565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.152024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd45-a045-437b-b9d7-aa14a20ee61a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.157509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 129/268] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/role_assignments?user.id=a729c8d183dc46c68984ba990ceb7036&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.158605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc99ce-ae88-4084-a486-f8504c4c7746 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.162440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc99ce-ae88-4084-a486-f8504c4c7746 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.162440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc99ce-ae88-4084-a486-f8504c4c7746 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.162440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 130/269] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:05.162440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc99ce-ae88-4084-a486-f8504c4c7746 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.169962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 140/270] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:04 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/0ccb0908e8ae45e084f562629dbd021d/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:05.185796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d528-0145-4aa6-992f-aecc0ce51d1d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.185993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d528-0145-4aa6-992f-aecc0ce51d1d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.186324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d528-0145-4aa6-992f-aecc0ce51d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.250457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f8b4d528-0145-4aa6-992f-aecc0ce51d1d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:05.252241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f8b4d528-0145-4aa6-992f-aecc0ce51d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:05.290294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.294567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.304259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc99ce-ae88-4084-a486-f8504c4c7746 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.337275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 141/271] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:05.338405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.338688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.338959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.356567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.357619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.358158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.358600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.360702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.361149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:05.362534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c21c5-57aa-442b-a069-731332f19c18 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.362640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:05.364897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 131/272] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.367584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.368107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.368554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.391665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:05.393039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:05.446029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 142/273] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:05.446819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.559788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 132/274] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:05.614659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.620420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86beb68-93a1-45c6-9570-0ddfc5a11aef None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.666391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.667095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.667653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.668016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.668952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=a729c8d183dc46c68984ba990ceb7036) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:05.669372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.671235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.680395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 143/275] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/role_assignments?user.id=a729c8d183dc46c68984ba990ceb7036&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.685188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.824444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.829280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521123a3-69ab-49bc-bf38-08448a3d6f3c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.845521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.887994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.888639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.888901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.889167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.900673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:05.905999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:05.906876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:05.907916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 133/276] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:05.914522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.998026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bcaac-bad4-4d32-b443-53b40affec9a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.039386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.040071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.040350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.040620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.048159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28350f6-20bb-4fd7-95ba-c8b73c0e94c2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.052265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:06.057983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:06.058853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:06.059654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:06.060733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 144/277] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:06.060733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.087466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.089996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.090601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.090876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.091145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.091800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:06.092131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.093516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.102532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 134/278] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.103246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.122349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.168406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fe059-9bef-4ec8-9ac0-44b77af77ab2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.204360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.205020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.205297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.205564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.206663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.208898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.210619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 145/279] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.211388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.218709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448eda14-a990-4470-98ac-81c7822b5faa None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.262064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.262663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.262938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.263204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.263880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:06.264179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.265563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.274525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 135/280] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.275325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.311260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.323523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c2de-014c-4d14-abd9-6073f9d83c85 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.360751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.361182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.361824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.362102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.362719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] PATH_INFO: `/v3/users/6fcee8f4bb2b47b392784b094610a274` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.364354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] RBAC: Authorizing `identity:get_user(user_id=6fcee8f4bb2b47b392784b094610a274)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.365982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.372229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 146/281] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/users/6fcee8f4bb2b47b392784b094610a274 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.372229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.398093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771532e5-d943-4842-b87e-08f0b876e60b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.439064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.439671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.440291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.440291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.440966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.442979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.481138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafe53c-a56b-45f0-95b1-ffbc0c906189 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.516911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.517538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.518708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.518708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] PATH_INFO: `/v3/users/278c217a6d3f46faa726daead58435af` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.531489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] RBAC: Authorizing `identity:get_user(user_id=278c217a6d3f46faa726daead58435af)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.532978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.534723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 147/282] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/users/278c217a6d3f46faa726daead58435af => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.535635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.554393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.587831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 136/283] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:06 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:06.588893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.663613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761bd61-c4e9-47fb-9cac-bc1bd3c96b16 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.714978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4de5-b769-4775-984a-e4c7322863f3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.718147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.719825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.720158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.720438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.721660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.723173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.726435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 148/284] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.731095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:06.760760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.767023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.767640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.767968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.768244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.768968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, user.id=278c217a6d3f46faa726daead58435af, scope.system=all) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:06.769275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.770808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98e9a-ce80-4e23-8146-9dc7b49aad16 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:06.776491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 137/285] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Aug 29 18:58:06 2022] GET /identity/v3/role_assignments?user.id=278c217a6d3f46faa726daead58435af&scope.system=all&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:06.853482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:06.869710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf2810-e3c4-49e5-9b97-1cfd7af62d3a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:06.908769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:06.909458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:06.909830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:06.910113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/6fcee8f4bb2b47b392784b094610a274/roles/5f31d1e396bf46d7a0ba15862b3d18ba` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:06.912222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, user_id=6fcee8f4bb2b47b392784b094610a274, role_id=5f31d1e396bf46d7a0ba15862b3d18ba)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:06.925243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837aa8c-d649-433e-b79a-7a6b0cc7ffd8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:07.016060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 149/286] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:06 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/users/6fcee8f4bb2b47b392784b094610a274/roles/5f31d1e396bf46d7a0ba15862b3d18ba => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:58:07.294045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051274d-9442-4892-bd68-116e1ee331bd None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:07.294537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051274d-9442-4892-bd68-116e1ee331bd None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:07.294537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051274d-9442-4892-bd68-116e1ee331bd None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:07.296210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 138/287] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:07.302507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:07.302793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:07.303157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:07.328930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:07.330133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:07.425427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 150/288] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:07.790628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051274d-9442-4892-bd68-116e1ee331bd None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:07.912587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051274d-9442-4892-bd68-116e1ee331bd None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:07.955296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:07.955968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:07.956252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:07.956524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:07.975641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:07.981064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:07.981970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:07.982772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:07.983381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 139/289] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:07.991817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.109522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.140690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae08b-ee4f-4bf5-a437-e8b2ee0111a7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.186217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.186828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.187104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.187381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.188071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:08.188378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.189740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.198254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 151/290] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:08.203779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.261951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.315063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0448943-7b28-4b69-8ba5-e5ee9f919912 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.361230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.361788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.362065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.362336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] PATH_INFO: `/v3/users/0ccb0908e8ae45e084f562629dbd021d` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.363568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] RBAC: Authorizing `identity:get_user(user_id=0ccb0908e8ae45e084f562629dbd021d)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.365307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.367056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 140/291] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/users/0ccb0908e8ae45e084f562629dbd021d => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:08.372505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.517748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed63a-3fad-4a47-a99b-0a3d83bde46f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.561705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.562705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.563189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.563692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.576815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.659809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.661301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.802170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 152/292] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:08.807970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.937336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc1a10-f465-42d3-92d2-5f36f4f7c10a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.968803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.988137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.988137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.988407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.988658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.989867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.991454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c297b-f6c3-4ccb-9ea7-8f26cd51c13b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.993431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 141/293] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:08.999415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.040730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.107789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d6af4-625d-44bf-8781-2d83255c6d2c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.118666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ee6f-8f16-446a-8528-85fac5fdf0a3 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.118945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ee6f-8f16-446a-8528-85fac5fdf0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.119222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ee6f-8f16-446a-8528-85fac5fdf0a3 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.120885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 142/294] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:09.125954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a1db-72b7-4671-8e5e-dce9af778565 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.126241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a1db-72b7-4671-8e5e-dce9af778565 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.126512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a1db-72b7-4671-8e5e-dce9af778565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.145614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c3c5a1db-72b7-4671-8e5e-dce9af778565 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:09.146948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c3c5a1db-72b7-4671-8e5e-dce9af778565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:09.156554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.157178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.157456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.157734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.158468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=0ccb0908e8ae45e084f562629dbd021d) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:09.158776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.160253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9212e1-4655-469d-9f77-456fd64668ab None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.174805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 153/295] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/role_assignments?user.id=0ccb0908e8ae45e084f562629dbd021d&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:09.256104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 143/296] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:09.301497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80f44d-cda9-4223-a29f-4d54241e46b9 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.302195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80f44d-cda9-4223-a29f-4d54241e46b9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.302447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80f44d-cda9-4223-a29f-4d54241e46b9 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.304185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 154/297] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:09.309880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f522300-3b7e-4545-952f-a24446c9c2a9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.310506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f522300-3b7e-4545-952f-a24446c9c2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.310995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f522300-3b7e-4545-952f-a24446c9c2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.337887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3f522300-3b7e-4545-952f-a24446c9c2a9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:09.339609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3f522300-3b7e-4545-952f-a24446c9c2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:09.435784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 144/298] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:09.446924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741109f-81b6-4559-9495-e75d7204a352 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.447212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741109f-81b6-4559-9495-e75d7204a352 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.447489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741109f-81b6-4559-9495-e75d7204a352 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.448903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 155/299] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:09.454291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737344-1316-4c76-8065-eac81e710a02 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.454651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737344-1316-4c76-8065-eac81e710a02 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.454999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737344-1316-4c76-8065-eac81e710a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.473449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-67737344-1316-4c76-8065-eac81e710a02 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:09.474889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-67737344-1316-4c76-8065-eac81e710a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:09.525465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.549845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 145/300] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:09.654227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741109f-81b6-4559-9495-e75d7204a352 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.668172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e590e6-2f0f-47de-baf6-a27b190e5c10 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.668479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e590e6-2f0f-47de-baf6-a27b190e5c10 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.668707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e590e6-2f0f-47de-baf6-a27b190e5c10 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.670412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 146/301] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:09.675668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.675980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.676256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.703285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:09.704935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:09.782553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 147/302] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:09.783433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.792943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741109f-81b6-4559-9495-e75d7204a352 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.794799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.838427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.839052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.839330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.839599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.851460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:09.856362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:09.857251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:09.858058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 156/303] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:09.858731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.896343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eb652-3c28-4d24-a131-6a6813c7a0c1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.909579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.936181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.936812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.937099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.937353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.947981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:09.953454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:09.954570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:09.955648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:09.958640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 148/304] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:09.959504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.972397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ad3-8834-43b6-b052-2e0f4a99334e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.012749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.013634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.013634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.013901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.024522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:10.029791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:10.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:10.031568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 157/305] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:10.032852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.060320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.074138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7c8c6-1742-4fe3-854f-a2c0c41fb9c4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.109345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.109944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.110221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.110491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.111168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.111469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.112885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.121534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 149/306] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.122348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.137691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2683d-e796-422b-bb2b-3a39ea9b1aa6 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.156018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.171250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.171870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.172152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.172424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.173086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.173386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.174771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.182610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 158/307] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.183391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.224145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263064d-c0b7-4268-8f58-5caa9bcede42 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.257062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.257652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.257923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.258189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.258861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.259155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.260544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.268297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.268522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 150/308] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.268522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.298055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfe794-6003-4bb7-b2f5-19caf493dd29 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.319953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.320583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.320857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.321128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.322235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.323673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.325264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 159/309] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.326027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.364650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.371320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e890-4849-4257-8746-bfc44d915df5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.406422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.407052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.407337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.407606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] PATH_INFO: `/v3/users/278c217a6d3f46faa726daead58435af` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.408756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] RBAC: Authorizing `identity:get_user(user_id=278c217a6d3f46faa726daead58435af)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.410253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.411848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 151/310] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/users/278c217a6d3f46faa726daead58435af => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.412628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.425354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066a6c-4bf2-41fa-b025-a278d6a857f4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.461547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.462183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.462460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.462728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] PATH_INFO: `/v3/users/7018dd6b077c4909a82426a37530095a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.476251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=7018dd6b077c4909a82426a37530095a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.477698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.479344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 160/311] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/users/7018dd6b077c4909a82426a37530095a => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.480116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.491993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.519962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bb654-df75-4be9-aef7-3cd237342929 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.554637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.555155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.555429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.555699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] PATH_INFO: `/v3/users/6fcee8f4bb2b47b392784b094610a274` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.557033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] RBAC: Authorizing `identity:get_user(user_id=6fcee8f4bb2b47b392784b094610a274)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.558481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.560072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 152/312] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/users/6fcee8f4bb2b47b392784b094610a274 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.560817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.579495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682b448-4074-4584-9a75-b0e72b34a0f5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.591450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.613696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.614844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.615179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.615530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] PATH_INFO: `/v3/groups` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.616189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.617622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.660539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9aac3-ae38-4a96-b917-33adcae9f39e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.693360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.694022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.694324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.694599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] PATH_INFO: `/v3/system/users/278c217a6d3f46faa726daead58435af/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.696045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=278c217a6d3f46faa726daead58435af, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.697434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.702018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 161/313] 10.222.0.62 () {60 vars in 1134 bytes} [Mon Aug 29 18:58:10 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:10.702846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.777452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 153/314] 10.222.0.62 () {58 vars in 1334 bytes} [Mon Aug 29 18:58:10 2022] PUT /identity/v3/system/users/278c217a6d3f46faa726daead58435af/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:10.780895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.786923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.824714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1385dc-f84a-4a31-88d9-4818a798feb5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.866584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.867668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.868169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.868642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.869609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, user.id=7018dd6b077c4909a82426a37530095a, scope.system=all) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.870127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.871740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.878898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 162/315] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/role_assignments?user.id=7018dd6b077c4909a82426a37530095a&scope.system=all&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.904043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.911460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30fd8-a165-4ad7-ad22-b11efb86ceea None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.957364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.958505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.958781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.959051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:11.044731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:11.046279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:11.149923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:11.162729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 154/316] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:11.164284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:11.272477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738411f-4515-4ada-a254-ca68ff364cf1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:11.310748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:11.311784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:11.312288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:11.312756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:11.314097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:11.315819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92328bb4-dc24-4f45-a4c5-62a2820940f3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:11.318691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 163/317] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:11 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:11.326940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:11.431121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c9dbb-75ca-4996-a9a0-36e7b19d0168 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:11.468327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:11.468940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:11.469217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:11.469493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:11.470221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79, user.id=6fcee8f4bb2b47b392784b094610a274) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:11.470529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:11.471996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fb21-012c-4c89-a832-9d37ae9ff62f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:11.483171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 155/318] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:11 2022] GET /identity/v3/role_assignments?user.id=6fcee8f4bb2b47b392784b094610a274&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:12.740999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2559-96b6-4539-9244-b9ed61389748 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.741535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2559-96b6-4539-9244-b9ed61389748 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.741535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2559-96b6-4539-9244-b9ed61389748 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.742997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 164/319] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:12.748052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece75f-ac03-4ed7-98f3-e32447c351fd None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.748427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece75f-ac03-4ed7-98f3-e32447c351fd None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.748589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece75f-ac03-4ed7-98f3-e32447c351fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.767321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9bece75f-ac03-4ed7-98f3-e32447c351fd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:12.768534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9bece75f-ac03-4ed7-98f3-e32447c351fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:12.772592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7868e-717f-47a0-84f3-b64705ba5d16 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.772865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7868e-717f-47a0-84f3-b64705ba5d16 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.773130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7868e-717f-47a0-84f3-b64705ba5d16 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.774451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 165/320] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:12.779422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.779703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.780001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.786207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:12.797783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:12.798952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:12.829555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:12.848501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 156/321] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:12.861016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad2d3-e685-4ab3-9091-beee8413eff3 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.861293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad2d3-e685-4ab3-9091-beee8413eff3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.861572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad2d3-e685-4ab3-9091-beee8413eff3 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.864017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 157/322] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:12.868368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.868990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.869652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.871765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 166/323] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:12.891045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:12.892273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:12.960316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 158/324] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:13.114966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.138302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.217741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1071c0-096b-41a7-b568-e7efa96d61a7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.248175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.250583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a22af-1fc4-405d-b22c-4a63aaca6a86 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.252953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.253589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.253862 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.254134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.264658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:13.268405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:13.269857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:13.270977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 167/325] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:13.271932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.284771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.285393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.285721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.285987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.287068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.288531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.290158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 159/326] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:13.290941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.374965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c456b9e-0f8a-48b8-bb9e-73a38983c029 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.389815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474599cd-784a-4d6d-b322-606eae2924a8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.408264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.408893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.409183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.409519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.419351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:13.422562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.423156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.423433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.423700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:13.424300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:13.425174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:13.425943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:13.427561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:13.427561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.427692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 168/327] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:13.427758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.427822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.437901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 160/328] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:13.438725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.480584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.489190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.527011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b858-f0bf-4543-8130-2a566bd01d16 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.539509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b22d4-0c3a-4b1e-8cc1-5912b6ced0f5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.563555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.564209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.564482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.564750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] PATH_INFO: `/v3/groups` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.565370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.566847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.573920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.574523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.574803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.575077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.575733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:13.576071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.577452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.585775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 161/329] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:13.586556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.656695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 169/330] 10.222.0.62 () {60 vars in 1134 bytes} [Mon Aug 29 18:58:13 2022] POST /identity/v3/groups => generated 226 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:13.657513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.691313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef98fd-d118-4108-8ae9-973dae79d32d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.705347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.737381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.738150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.738490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.738887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] PATH_INFO: `/v3/users/278c217a6d3f46faa726daead58435af` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.740868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] RBAC: Authorizing `identity:get_user(user_id=278c217a6d3f46faa726daead58435af)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.742459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.749131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 162/331] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/users/278c217a6d3f46faa726daead58435af => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:13.750306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.789938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959578-fe49-45c7-8f7b-7658e4773f2d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.805310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.832536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.833180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.833460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.833729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] PATH_INFO: `/v3/users/7018dd6b077c4909a82426a37530095a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.835312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] RBAC: Authorizing `identity:get_user(user_id=7018dd6b077c4909a82426a37530095a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.837432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.839436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 170/332] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/users/7018dd6b077c4909a82426a37530095a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:13.844924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.877217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70adf-a1e5-47ba-9e45-3dc6e5cee875 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.922078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.922977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.923247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.926414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] RBAC: Adding query filter params (role.id=f1db7b15f40a4f39bce1393968346f60, user.id=278c217a6d3f46faa726daead58435af, scope.system=all) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:13.926414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.926414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffde0f-f851-4579-a286-2e2c78b1070c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.940367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 163/333] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/role_assignments?user.id=278c217a6d3f46faa726daead58435af&scope.system=all&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:13.977259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d8599-d2da-4f61-9c80-0f35e93746b0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:14.017861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:14.018539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:14.018830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:14.019092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] PATH_INFO: `/v3/system/users/7018dd6b077c4909a82426a37530095a/roles/5f31d1e396bf46d7a0ba15862b3d18ba` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:14.021204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7018dd6b077c4909a82426a37530095a, role_id=5f31d1e396bf46d7a0ba15862b3d18ba)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:14.022611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a2d55-c526-4ce5-9bf6-cc632db7f170 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:14.091458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 171/334] 10.222.0.62 () {58 vars in 1334 bytes} [Mon Aug 29 18:58:13 2022] PUT /identity/v3/system/users/7018dd6b077c4909a82426a37530095a/roles/5f31d1e396bf46d7a0ba15862b3d18ba => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:58:15.774684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef425e5e-a538-48fe-b406-d48726478a18 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:15.775702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef425e5e-a538-48fe-b406-d48726478a18 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:15.776207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef425e5e-a538-48fe-b406-d48726478a18 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:15.777746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 164/335] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:15.784195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeab252-474f-49c2-bd9c-2c4b06124ac3 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:15.785087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeab252-474f-49c2-bd9c-2c4b06124ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:15.786080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeab252-474f-49c2-bd9c-2c4b06124ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:15.807375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8eeab252-474f-49c2-bd9c-2c4b06124ac3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:15.808953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8eeab252-474f-49c2-bd9c-2c4b06124ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:15.859335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:15.890119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 172/336] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:16.078296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af04-a6bd-4343-832b-21057e5128d5 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.078296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af04-a6bd-4343-832b-21057e5128d5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.078296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af04-a6bd-4343-832b-21057e5128d5 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.078980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 165/337] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:16.085311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef002d-43c5-47ef-a061-708dd9739096 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.085812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef002d-43c5-47ef-a061-708dd9739096 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.086274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef002d-43c5-47ef-a061-708dd9739096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.107271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-14ef002d-43c5-47ef-a061-708dd9739096 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:16.108809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-14ef002d-43c5-47ef-a061-708dd9739096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:16.197175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 173/338] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:16.272966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7f0b0-80f8-4206-82e3-35459980acc6 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.273566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7f0b0-80f8-4206-82e3-35459980acc6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.274052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7f0b0-80f8-4206-82e3-35459980acc6 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.275583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 166/339] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:16.281004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.281586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.282058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.308856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:16.310400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:16.322161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7f0b0-80f8-4206-82e3-35459980acc6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:16.391822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:16.406422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 174/340] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:16.490106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7f0b0-80f8-4206-82e3-35459980acc6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:16.530937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:16.531880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.532354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.532847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.547773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:16.553754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:16.554612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:16.555386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:16.557718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 167/341] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:16.563738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:16.615883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:16.690205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21900063-ad98-4b4d-9a5a-b88042773ad8 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:16.698728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:16.798429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:16.799664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:16.800101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:16.800511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:16.801453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:16.801912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:16.805198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:16.829906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 175/342] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:16.836072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:16.948084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704911d3-bb32-438e-aec2-fcdd0553c687 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:16.952148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:16.999986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.001395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.002030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.002302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.002567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.010919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e7c33-2883-4733-a5a5-f42b966beae0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.026279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:17.039240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:17.040284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:17.041044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:17.042328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 168/343] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:17.042966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.085176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.085798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.086071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.086335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.174312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.176893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904fb7-81bb-44eb-8205-de301eb63728 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.188760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590ec9-52b2-4c9f-a809-94e4c44decf4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.228922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.229477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.229751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.230064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.249294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:17.257822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:17.258755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 169/344] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:17.260058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.289715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.292746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.299805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 176/345] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:58:17.301773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758b06-438e-466d-b187-77799e50b43d None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.302733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758b06-438e-466d-b187-77799e50b43d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.302733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758b06-438e-466d-b187-77799e50b43d None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.303999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 177/346] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:17.304886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758b06-438e-466d-b187-77799e50b43d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.399443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb6085-fe94-421c-86dc-d9da45696320 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.431988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758b06-438e-466d-b187-77799e50b43d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.452296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.453506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.453977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.454492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.455451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:17.456100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.457720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.468179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 170/347] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:17.469303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.478694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.479378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.479585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.479869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.480545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:17.480857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.482791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc478ad-5048-4018-a5b6-b4e6a3474cf3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.495178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 178/348] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:17.497487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.497964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.498424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.521959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:17.523431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:17.583043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.590882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.606421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5f072-4193-40f5-a848-371da8e6bdb4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.609077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 179/349] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:17.610122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.649191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.650225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.650751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.651331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.652884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.654721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.656838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 171/350] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:17.657833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.746776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f10e-71c9-42af-a7a3-fe082be0a190 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.788349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271275a-155a-4345-840b-8b3a9377feae None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:17.796981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.797604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.797904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.798181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.827106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:17.827736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.828056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.828327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] PATH_INFO: `/v3/users/7018dd6b077c4909a82426a37530095a` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.829455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] RBAC: Authorizing `identity:get_user(user_id=7018dd6b077c4909a82426a37530095a)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.830909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.833607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 172/351] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/users/7018dd6b077c4909a82426a37530095a => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:17.834478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:17.859971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.877899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:17.892332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:17.893850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066da238-b67b-4395-940e-ec6f51a824a4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:17.983897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f2f40-2a66-4260-9548-50869a43ad29 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.022386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 180/352] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:18.023594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf35cb-3228-464b-89e4-b67382e51ef8 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.025978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.026646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.026921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.027195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.028883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf35cb-3228-464b-89e4-b67382e51ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.029057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf35cb-3228-464b-89e4-b67382e51ef8 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.030594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 181/353] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:18.031173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf35cb-3228-464b-89e4-b67382e51ef8 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.036148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.039055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.043887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 173/354] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:17 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.043887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.144981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.149550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.163943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf35cb-3228-464b-89e4-b67382e51ef8 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.172387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f30a-45bd-482e-bb82-55cff12be276 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.214752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.215372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.215681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.215936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.216661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] RBAC: Adding query filter params (role.id=5f31d1e396bf46d7a0ba15862b3d18ba, user.id=7018dd6b077c4909a82426a37530095a, scope.system=all) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:18.216965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.218030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.218412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.218735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.219018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.219286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.234355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:18.235922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 182/355] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/role_assignments?user.id=7018dd6b077c4909a82426a37530095a&scope.system=all&role.id=5f31d1e396bf46d7a0ba15862b3d18ba => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.236842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-07da4ca1-13ba-490a-b2eb-d8823f4fe531 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:18.241707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:18.242651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 174/356] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:18.248331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.249066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.249066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.266703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:18.268084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:18.360037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 175/357] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:18.360985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.388268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ca819-0f58-40ee-87e4-b90fa3571a9b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.441140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.442453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.442714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.443208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.444410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.445865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.447501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 183/358] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.448294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.456586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.467760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.497206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac388a9-3905-4ba3-9846-1bad6cd9341e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.538217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.538829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.539103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.539370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.540135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:18.540442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.541964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19859171-e717-4161-adb0-1e67e979e2d5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.552843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 176/359] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.554173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53852aaf-803d-4433-811d-f1a87e0f4660 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.554459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53852aaf-803d-4433-811d-f1a87e0f4660 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.554730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53852aaf-803d-4433-811d-f1a87e0f4660 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.557749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 177/360] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:18.558578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53852aaf-803d-4433-811d-f1a87e0f4660 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.579601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934e24d-1073-40dd-b275-15b1a5f0a795 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.629134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.629134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.630720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.631024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.631747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:18.632078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.633495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3927-66cd-4c8e-838f-4896772df17d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.650005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 184/361] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.650005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.650005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.650005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.677946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:18.679690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:18.740406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53852aaf-803d-4433-811d-f1a87e0f4660 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.768651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 185/362] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:18.769897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.775784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.786535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.795267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.795978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.796259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.796530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.797682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.799561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.802707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 178/363] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.803496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.897137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68210723-1c90-48a5-9ae3-6d8e0dc784c2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.936746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.937352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.937629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.937993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.953095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba18d0-fae6-4d15-9ffb-381cd326368e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.994650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.995330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.996141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.996423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.010884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:19.017038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:19.017910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:19.018716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:19.022073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 179/364] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:19.028315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.040463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.042234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.051617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.104126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.164608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc9a2-2815-4a9b-ae09-8be63a9e8303 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.179679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 186/365] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:19.179679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.201471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.202084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.202363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.202634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.203357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:19.203666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.205163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.211075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 180/366] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:19.215938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.291508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0cdab-df46-4112-9812-5ba6b5a1f7c2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.301930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.332246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcb36-900d-4680-8209-0feb2d11e2a7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.333789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.334524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.334807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.335228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.335938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:19.336245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.337630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.348166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 187/367] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:19.349375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.365528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.384910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.385832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.386116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.386394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.404635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:19.412024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:19.413132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:19.414110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:19.414922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 181/368] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:19.416051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.476532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636b9e8-bd77-45d1-886d-c5f6dd889917 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.525403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.527354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.527828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.528318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.529643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.531265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.534412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 188/369] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:19.536398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.559741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c095cc-b4d2-4772-839a-860b00c8d99a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.580262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.612681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.613679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.613962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.614236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.668505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7521b34-b482-42ff-9358-fa85c1ef7662 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.695779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.712151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.713489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.713974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.714433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.715460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:19.716014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.718166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.723020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.725055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.731106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 189/370] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:19.732221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.856053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 182/371] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:19.857322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.869265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.873371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df2af8-ac47-40df-aa03-a75010616c6d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.908100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.909523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.910027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.911125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.913558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.915209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.917365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 190/372] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:19.918340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:19.952741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:19.969586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd4ae3-e1e1-4bc5-b851-f16677a16e89 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.025389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.027639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.028089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.028467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.040828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0372a-c025-4d34-932f-a3c6763283e3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.078500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.079389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.079805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.080240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.081574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.083813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.086085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 191/373] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.087115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.112685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.114420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.124088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.196212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77999df2-d52b-4f8c-a00b-70b7078fe180 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.208844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.227745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 183/374] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:20.233183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.237263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.238072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.238975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.239339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.240269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:20.240678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.242204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.248457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 192/375] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.255175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.349841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d401f5-052e-4ae0-9c19-566c16c3c890 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.392167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.392836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.393115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.393392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.394581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.396865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.398518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 184/376] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.404631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.470107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.485576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f83df-81df-47ea-b4d0-ca22986f0f3b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.503277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.524266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe73af-7030-44d0-b700-f910d506d29f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.532018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.532771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.533057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.533332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] PATH_INFO: `/v3/projects/bb936e104dc24a75a45f4d04495d191b` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.534499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=bb936e104dc24a75a45f4d04495d191b)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.536518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.538584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 193/377] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/projects/bb936e104dc24a75a45f4d04495d191b => generated 344 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.547393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.568388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.569008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.569279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.569552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.570304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:20.570611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.572122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.581448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 185/378] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:20.704540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0475c-319a-4dee-a7cf-b6345cbf4dd4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.748352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.748917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.749190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.749456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] PATH_INFO: `/v3/groups/99cf73a8c32a4fa69260222e17bd20b2` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.756675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] RBAC: Authorizing `identity:get_group(group_id=99cf73a8c32a4fa69260222e17bd20b2)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.758138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf0cf5-c0af-41f4-a1a2-e764ac32ba67 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.759802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 194/379] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/groups/99cf73a8c32a4fa69260222e17bd20b2 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.765675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:20.824816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e691a-dcd2-4db3-82af-ae3cff6c3e1e None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.825146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e691a-dcd2-4db3-82af-ae3cff6c3e1e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.825360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e691a-dcd2-4db3-82af-ae3cff6c3e1e None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.826931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 195/380] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:20.832023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.832827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.833095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.856365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:20.857555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:20.881242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4d28-0df7-4d61-92e4-f19c8bbb752e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:20.894324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.899407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:20.918743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:20.919356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:20.919633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:20.919964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:20.920695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] RBAC: Adding query filter params (group.id=99cf73a8c32a4fa69260222e17bd20b2, role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=bb936e104dc24a75a45f4d04495d191b) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:20.921033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:20.922462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:20.929170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 186/381] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:20 2022] GET /identity/v3/role_assignments?group.id=99cf73a8c32a4fa69260222e17bd20b2&scope.project.id=bb936e104dc24a75a45f4d04495d191b&role.id=ae077013d56245c7869e76158c81fd53 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:20.960697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 196/382] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:21.313006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.437577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd763db-5380-40a7-9a4d-4fd4fffc660b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.479375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.480070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.480350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.480624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.494177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:21.501357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:21.502241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:21.504631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 187/383] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:21.509097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.665733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be356b2-de3b-4017-82e5-5979d689260a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.733483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.734112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.734393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.734666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.735335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:21.735638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:21.737074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1d3a2-2cba-4dc5-9223-33794134dc9d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:21.747344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 197/384] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:21.753389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.824790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:21.864936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72f8a-790d-43ff-a77d-bca0255ab3dd None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.865237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72f8a-790d-43ff-a77d-bca0255ab3dd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.865481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72f8a-790d-43ff-a77d-bca0255ab3dd None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.867042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 198/385] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:21.879668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.880001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.880255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.903313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10506f-382e-4a3d-a878-b42afe22e2b6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.912888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:21.914096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:21.934974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:21.945841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.946481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.946755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.947023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.952637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:21.952637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:21.952637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 188/386] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:21.958356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:22.044882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 199/387] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:22.076487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef81b2-51da-490f-9468-42101a64b858 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:22.086356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:22.122637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:22.124656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.125175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.125651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.128012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:22.130252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:22.132111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 189/388] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:22.137948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:22.253404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3d7d-0612-4f7e-83f6-0adb919313ec None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:22.296333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:22.297434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.297915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.298405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/groups/7111f732984941f9a2ec80b7bc6ca705/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.304009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, group_id=7111f732984941f9a2ec80b7bc6ca705, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:22.305680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:22.383982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:22.409280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 200/389] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Aug 29 18:58:22 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/groups/7111f732984941f9a2ec80b7bc6ca705/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:22.545975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb36f-2de0-44f1-8ce4-9555f80005e7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:22.585505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:22.586497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.586976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.587444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.599593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:22.605686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:22.606561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:22.607358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:22.609674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 190/390] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:22.616469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:22.635004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:22.731019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb9778-0028-4989-a833-f5bd2ee6c9a6 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:22.768822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:22.770453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.770967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.771451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.772333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:22.772822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:22.774432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02b550-37ef-4ebf-a40f-d13a96b5daa3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:22.783493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 201/391] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:22.790109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:22.856109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:22.874755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476885-bf56-4600-bf98-c742c647d9e2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.875359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476885-bf56-4600-bf98-c742c647d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.875821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476885-bf56-4600-bf98-c742c647d9e2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.877437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 202/392] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:22.883550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:22.884191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:22.884676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:22.905910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:22.907406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:22.951176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b82718-7f5e-433e-89d4-40e1f261ea96 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:22.990925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 203/393] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:23.003404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.004418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.004916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.005421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.012557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:23.014287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:23.016278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 191/394] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:22 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:23.023177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.027038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:23.154840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18f4a6-a4a3-4981-a9bc-18ae262c1a8f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:23.198569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.199572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.200105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.200575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.201947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:23.203573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3163-315a-4a4c-be51-ac317eadbfc0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:23.206678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 204/395] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:23.212249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.334815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce7de8-af3f-429d-b75b-ced6cec2bdca None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:23.340934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:23.348276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccd40c-cd90-4cf4-9bd9-603d5026e3df None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.348568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccd40c-cd90-4cf4-9bd9-603d5026e3df None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.349113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccd40c-cd90-4cf4-9bd9-603d5026e3df None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.350772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 205/396] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:23.354856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccd40c-cd90-4cf4-9bd9-603d5026e3df None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.383634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.384335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.384865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.384865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/groups/7111f732984941f9a2ec80b7bc6ca705/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.387169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] RBAC: Authorizing `identity:create_grant(project_id=58a1723a48ab4d00814684dd4c6b9daa, group_id=7111f732984941f9a2ec80b7bc6ca705, role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:23.388770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13081f-2119-4758-9534-2447662b4b4b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:23.444005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:23.475979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 192/397] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Aug 29 18:58:23 2022] PUT /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa/groups/7111f732984941f9a2ec80b7bc6ca705/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:23.475979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fff3dc-abae-4b82-93b6-38e947b9e4f9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.475979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fff3dc-abae-4b82-93b6-38e947b9e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.475979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fff3dc-abae-4b82-93b6-38e947b9e4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.491161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccd40c-cd90-4cf4-9bd9-603d5026e3df None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:23.496111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-81fff3dc-abae-4b82-93b6-38e947b9e4f9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:23.497615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-81fff3dc-abae-4b82-93b6-38e947b9e4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:23.543438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.544139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.544417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.544687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.560714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-804ea62e-2907-4855-a8f5-e5becd5a2178 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:23.566586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:23.567494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:23.568330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:23.572782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 206/398] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:23.574872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3598-fc39-463b-bc62-0f6a361e5071 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.575084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3598-fc39-463b-bc62-0f6a361e5071 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.575317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3598-fc39-463b-bc62-0f6a361e5071 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.576951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 207/399] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:23.577707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3598-fc39-463b-bc62-0f6a361e5071 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.594988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 193/400] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:23.596777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.597307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.598353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.619489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:23.620772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:23.693843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 194/401] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:23.708651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3598-fc39-463b-bc62-0f6a361e5071 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:23.762924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:23.774361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.774980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.775250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.775518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.776213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] RBAC: Adding query filter params (name=member) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:23.776512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:23.777882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:23.791110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 208/402] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:23.796666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.837996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:23.931047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8598a50-3129-4de0-8a54-708c5264dbe4 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:23.935797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:23.969807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:23.970777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:23.971061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:23.971334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:23.974402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:23.975304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:23.976933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 195/403] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:23.983414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.088474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:24.096742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399bfdd-da26-4eee-bf23-691751b11af8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.134083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.134725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.134998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.135267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.136474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:24.138051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:24.140101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 196/404] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:24.140976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.158723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e7c3-7845-4181-a171-7610e1477ede None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.198924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.199562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.199841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.200296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.214141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:24.221550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:24.223011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 209/405] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:24.228547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.266526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc416-2535-413e-9633-755a3d75b588 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.298376 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:24.305421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.306395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.306870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.312264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.319548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:24.324779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:24.325684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:24.326475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:24.329129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 197/406] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:24.330213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.353768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0dd14-1813-4356-99fc-e6c2dad5a9e7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.359847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:24.390897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.391837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.392365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.393533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/groups/7111f732984941f9a2ec80b7bc6ca705/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.396708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, group_id=7111f732984941f9a2ec80b7bc6ca705, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:24.398461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:24.446982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6245b5-c83c-4aa8-a904-7d5488ec4ed2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.481693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.482305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.482586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.482855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.483508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:24.483808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:24.485292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:24.499209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 198/407] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:24.499209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.516481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 210/408] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Aug 29 18:58:24 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/groups/7111f732984941f9a2ec80b7bc6ca705/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:24.520037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.692027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:24.707999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fff06-03b6-4325-a096-d81b827f3cfb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.726462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:24.727675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9e0f2-c2da-4ebc-93a5-a9865e314467 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:24.759222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.760241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.760734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.761208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.766614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:24.768579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:24.771068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 211/409] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:24.777110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.779043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:24.779043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:24.779360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:24.779570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:24.780272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:24.780576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:24.781956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:24.795967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 199/410] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:24.799884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:24.912664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bcb0a-55cb-4f79-960b-c649d4c836e5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:25.009923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:25.012035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6349e4d-539c-49bf-80fb-fc51bf512063 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:25.012889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.013344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.013791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.016156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:25.018390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:25.027963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 212/411] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:25.034007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:25.092267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:25.093680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.095212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.096384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] PATH_INFO: `/v3/groups/99cf73a8c32a4fa69260222e17bd20b2` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.099893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] RBAC: Authorizing `identity:get_group(group_id=99cf73a8c32a4fa69260222e17bd20b2)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:25.101734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:25.103909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 200/412] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:24 2022] GET /identity/v3/groups/99cf73a8c32a4fa69260222e17bd20b2 => generated 229 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:25.110374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:25.134210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:25.135639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:25.198050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b40d-aca3-4bec-9293-781e171d50ea None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:25.242115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:25.242813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.243093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.243692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/groups/7111f732984941f9a2ec80b7bc6ca705/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.246312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d9095131e34a2a8c37fe99e4193c79, group_id=7111f732984941f9a2ec80b7bc6ca705, role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:25.248393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e159c-270a-43b2-9335-5dc33e602255 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:25.253504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d6ba6-b917-4bb5-b63f-a3eb3dbb3ecd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:25.290352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:25.291336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.291813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.292331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] PATH_INFO: `/v3/projects/bb936e104dc24a75a45f4d04495d191b` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.293648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] RBAC: Authorizing `identity:get_project(project_id=bb936e104dc24a75a45f4d04495d191b)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:25.295474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b836f90-e6f3-44aa-b0e1-2dc19e26e304 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:25.297406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 201/413] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:25 2022] GET /identity/v3/projects/bb936e104dc24a75a45f4d04495d191b => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:25.298831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c550aec-ff35-4e46-88d7-85368eb32488 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.299255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c550aec-ff35-4e46-88d7-85368eb32488 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.299726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c550aec-ff35-4e46-88d7-85368eb32488 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.301880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 202/414] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:25.303356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c550aec-ff35-4e46-88d7-85368eb32488 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:25.338134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 213/415] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Aug 29 18:58:25 2022] PUT /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79/groups/7111f732984941f9a2ec80b7bc6ca705/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:25.338134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ce8dc-cadf-4aed-b3fd-42e19dedb452 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.338134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ce8dc-cadf-4aed-b3fd-42e19dedb452 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.338134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ce8dc-cadf-4aed-b3fd-42e19dedb452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.359621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-3b4ce8dc-cadf-4aed-b3fd-42e19dedb452 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:25.359621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3b4ce8dc-cadf-4aed-b3fd-42e19dedb452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:25.465009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:25.508432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 214/416] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:25.514821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c550aec-ff35-4e46-88d7-85368eb32488 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:25.564038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:25.564626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.564899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.565165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] PATH_INFO: `/v3/projects/bb936e104dc24a75a45f4d04495d191b/groups/99cf73a8c32a4fa69260222e17bd20b2/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.567605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb936e104dc24a75a45f4d04495d191b, group_id=99cf73a8c32a4fa69260222e17bd20b2, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:25.569127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e3ed-e627-4a61-a4b2-3414637d3346 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:25.639829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 203/417] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Aug 29 18:58:25 2022] PUT /identity/v3/projects/bb936e104dc24a75a45f4d04495d191b/groups/99cf73a8c32a4fa69260222e17bd20b2/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:25.733575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbf84d-60e8-4096-a21a-58501db79178 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.734209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbf84d-60e8-4096-a21a-58501db79178 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.734726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbf84d-60e8-4096-a21a-58501db79178 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.736838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 215/418] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:25.743017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:25.743315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:25.743561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:25.773030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:25.774253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:25.854454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbf84d-60e8-4096-a21a-58501db79178 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:25.861107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 204/419] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:25.907951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:25.984507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbf84d-60e8-4096-a21a-58501db79178 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.036711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.037350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.037625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.037891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.052114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:26.056974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:26.057833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:26.058659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 216/420] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:26.065159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.145275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.208258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:26.245580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21491-9f06-4b87-9c16-a5f3a9cde3f5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.318459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.319131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.319409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.319793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.324045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:26.324363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:26.325838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:26.328648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:26.332033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c630f0c-fdc4-42b2-98c3-57cad33a4007 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.349997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 205/421] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:26.360942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.385856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.386498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.386773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.387445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.407234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:26.417535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:26.418490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:26.419324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:26.424570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 217/422] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:26.428550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.539431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ea59-199c-4146-9796-b636b6b90f48 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.573516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46375652-b412-467e-ba84-c889a1634aa9 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.597845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:26.606420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.610903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.610903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.610903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.622272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.622885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.623157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.623424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.624116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:26.624411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:26.625803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:26.639070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 218/423] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:26.645296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.700209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:26.750047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:26.752825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498b38b-3d60-491c-b818-4770ccc1f16d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:26.835306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611545ce-2df6-4e6f-8ecc-aa18b93e5b14 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:26.870611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:26.871218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.871491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.871760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.912513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 206/424] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:26.913873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f575a-65db-40e5-ae49-88a0173a6d5d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:26.914162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f575a-65db-40e5-ae49-88a0173a6d5d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:26.914438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f575a-65db-40e5-ae49-88a0173a6d5d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:26.915952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 207/425] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:26.922779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f575a-65db-40e5-ae49-88a0173a6d5d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:26.952997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:26.955110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b1ede-ebe9-4cfa-90fd-aae4e5bffb37 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:26.958496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:26.993408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.040841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f575a-65db-40e5-ae49-88a0173a6d5d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.080368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.081171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.081935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.083354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.085086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.086836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 219/426] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:27.088194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.088476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.088746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.091838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.094131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 208/427] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:26 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:27.095613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.116559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:27.118194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:27.202229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 220/428] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:27.203188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.265143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.271272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.277715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17048542-ab0b-40fe-a52d-010d04d28557 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.335270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff1988-7ad0-4579-bb29-2210e18e44a7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.351220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.351882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.352146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.352412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] PATH_INFO: `/v3/projects/58a1723a48ab4d00814684dd4c6b9daa` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.353635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] RBAC: Authorizing `identity:get_project(project_id=58a1723a48ab4d00814684dd4c6b9daa)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.355076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.356902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 209/429] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/projects/58a1723a48ab4d00814684dd4c6b9daa => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:27.365569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.406625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.407261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.407535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.407803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.409114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.412043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.414702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 221/430] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:27.420561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.561120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ef66-30d8-4fd3-9510-2c20fd4e72d8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.579147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6000c4f-9e1d-4f24-9001-6ac5e5b7f462 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.600960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.601703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.602043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.602321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.603478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.604401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.605435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.607149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 210/431] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:27.612537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.619317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.620239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.620239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.620491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.621618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:27.621929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.623723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4015aa-943c-4355-acef-be64783b6f46 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 222/432] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b4f6-44fd-4bb2-a02c-4dd33d690dd0 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b4f6-44fd-4bb2-a02c-4dd33d690dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b4f6-44fd-4bb2-a02c-4dd33d690dd0 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 223/433] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:27.635899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b4f6-44fd-4bb2-a02c-4dd33d690dd0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.656127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.751632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b010-2cb8-4e94-a5aa-f43f68454a81 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.766158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b4f6-44fd-4bb2-a02c-4dd33d690dd0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.802834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.803504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.803793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.804099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.806275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:27.806889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.807151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.807422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.808849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=58a1723a48ab4d00814684dd4c6b9daa) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:27.809943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:27.811309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f341e09-4ee6-48f1-bb6b-1db258e3c4c3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:27.816220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:27.817692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 224/434] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=58a1723a48ab4d00814684dd4c6b9daa&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:27.819519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c2c87-ae06-41ae-a342-67a1769bf135 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.819803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c2c87-ae06-41ae-a342-67a1769bf135 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.820095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c2c87-ae06-41ae-a342-67a1769bf135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:27.822430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:27.823303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:27.824120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 211/435] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:27.829470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:27.844513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ad1c2c87-ae06-41ae-a342-67a1769bf135 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:27.845717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ad1c2c87-ae06-41ae-a342-67a1769bf135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:27.902500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.932081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 225/436] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:27.963171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:27.968718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2021-df00-4e52-9618-c0e16e12fa3d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:27.984845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e139-2ced-49d9-b615-b48e3b307e10 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.985159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e139-2ced-49d9-b615-b48e3b307e10 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.985394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e139-2ced-49d9-b615-b48e3b307e10 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:27.991886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 226/437] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:27.993549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:27.993826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:27.994097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.015981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.016339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:28.016599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.018419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:28.018535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.018535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.018535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] RBAC: Adding query filter params (name=member) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:28.018731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.020281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.030306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 212/438] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:58:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:28.048627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.109278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 227/439] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:28.180457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119367a-73eb-47da-bc0d-46aab228578b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.216962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.217564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.217840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.218122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.235714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.252610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.317306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.319057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.371618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db9ee2-7e6b-4f77-8c5f-7788119636b3 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.388299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.413120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.414028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.414028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.414297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.428428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:28.433750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:28.435871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:28.436676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:28.440276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 228/440] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:28.441745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.451038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 213/441] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:28.451679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.522703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.555961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986aee3-10e1-4d48-880b-11d93fcd8479 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.565320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcdb4d-de7b-4c6a-bf81-8f9e6675d529 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.588896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.589528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.589808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.590081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.599200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.599846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.600164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.600577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.601129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:28.601227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:28.601517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.602880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:28.606143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:28.606998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:28.607787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 229/442] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:28.608472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.612128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 214/443] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:28.616831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.638190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.713764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c9f9-720e-4073-8d3e-f9dbdf95cf3b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.717042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bd2c-c74e-480d-bd25-d2a3b1617efa None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.737079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.747517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.748168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.748440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.749975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.749975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.751012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.751299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.751613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.751918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.752193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.752939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 230/444] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:28.754250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.826228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.827709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.847829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:28.853861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff92c-c206-4bd0-ba86-a600e1f75540 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:28.887540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:28.888171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:28.888495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:28.888718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:28.889371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:28.889673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:28.891028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:28.899621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 231/445] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:28.900251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.927187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 215/446] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:58:28.928411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:28.951171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.006208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d45624-be12-47f8-9291-cf41e17a2574 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.031921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d73d44-3b91-46bb-932c-fb654445400c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.039706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.040439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.040720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.040995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.042161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.043609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.045210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 232/447] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:29.045982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.068713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.069314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.069590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.069861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.074680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.137313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.138832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.146595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361f50b-c866-49d3-83fe-e9f8b838f076 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.159132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.181777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.182411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.182688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.182963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] PATH_INFO: `/v3/projects/c6d9095131e34a2a8c37fe99e4193c79` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.184106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d9095131e34a2a8c37fe99e4193c79)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.185606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.187231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 233/448] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/projects/c6d9095131e34a2a8c37fe99e4193c79 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:29.188017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.237010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 216/449] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:29.237869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.285124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.291339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d60c-3495-4134-81cd-fc3db8e432d7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.324861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.325516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.326201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.326476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.327196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=f1db7b15f40a4f39bce1393968346f60, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:29.327501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.328949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.335128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 234/450] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=f1db7b15f40a4f39bce1393968346f60 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:29.335947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.338005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c73d-2d4c-44bf-b4ba-730365a2139e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.360042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.375937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.376583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.376864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.377139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] PATH_INFO: `/v3/groups/7111f732984941f9a2ec80b7bc6ca705` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.378309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] RBAC: Authorizing `identity:get_group(group_id=7111f732984941f9a2ec80b7bc6ca705)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.380656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.387363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 217/451] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/groups/7111f732984941f9a2ec80b7bc6ca705 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:29.388626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.463179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4597-0041-4922-8ee7-48d50f60a993 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.502407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0aa5-10b0-44b1-bda0-dded9813b21c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.505957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.506768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.507106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.507380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] PATH_INFO: `/v3/projects/bb936e104dc24a75a45f4d04495d191b` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.508825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] RBAC: Authorizing `identity:get_project(project_id=bb936e104dc24a75a45f4d04495d191b)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.510205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.511925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 235/452] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/projects/bb936e104dc24a75a45f4d04495d191b => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:29.517187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.544672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.545289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.545575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.545848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.546571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] RBAC: Adding query filter params (group.id=7111f732984941f9a2ec80b7bc6ca705, role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=c6d9095131e34a2a8c37fe99e4193c79) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:29.546880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.548409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.548760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.556586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 218/453] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/role_assignments?group.id=7111f732984941f9a2ec80b7bc6ca705&scope.project.id=c6d9095131e34a2a8c37fe99e4193c79&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:29.634912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee973-f1fd-44de-8eb5-e8962bde51ed None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.671803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.672488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.672766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.673049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] PATH_INFO: `/v3/groups/99cf73a8c32a4fa69260222e17bd20b2` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.674240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] RBAC: Authorizing `identity:get_group(group_id=99cf73a8c32a4fa69260222e17bd20b2)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.676391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfb43-9d52-4e47-8817-ef7f4a038a45 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.678759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 236/454] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/groups/99cf73a8c32a4fa69260222e17bd20b2 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:29.685151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:29.746569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:29.794655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd80f7-84a3-4639-a4ea-25a96555b532 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:29.829125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:29.829727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:29.830010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:29.830279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:29.830997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] RBAC: Adding query filter params (group.id=99cf73a8c32a4fa69260222e17bd20b2, role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=bb936e104dc24a75a45f4d04495d191b) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:29.831306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:29.833326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cc8c4-fe8c-437e-8860-31eaa63c93e8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:29.839192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 219/455] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Aug 29 18:58:29 2022] GET /identity/v3/role_assignments?group.id=99cf73a8c32a4fa69260222e17bd20b2&scope.project.id=bb936e104dc24a75a45f4d04495d191b&role.id=ae077013d56245c7869e76158c81fd53 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:32.084184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e19a8-4da8-41e9-9efc-226eb09e3b54 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.085132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e19a8-4da8-41e9-9efc-226eb09e3b54 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.085627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e19a8-4da8-41e9-9efc-226eb09e3b54 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.087425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 237/456] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:32.093271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d50ee-0c6c-4109-90d7-fd5c4aef78dd None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.093705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d50ee-0c6c-4109-90d7-fd5c4aef78dd None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.093808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d50ee-0c6c-4109-90d7-fd5c4aef78dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.121093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8b0d50ee-0c6c-4109-90d7-fd5c4aef78dd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:32.122303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8b0d50ee-0c6c-4109-90d7-fd5c4aef78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:32.205628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 220/457] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:32.431650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c8669-0456-424d-a315-d078a4b3a1e1 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.432334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c8669-0456-424d-a315-d078a4b3a1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.432819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c8669-0456-424d-a315-d078a4b3a1e1 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.434628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 238/458] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:32.439507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.439784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.440079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.464787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:32.465973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:32.517160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:32.548196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 221/459] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:32.584589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c8669-0456-424d-a315-d078a4b3a1e1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:32.700348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:32.713496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c8669-0456-424d-a315-d078a4b3a1e1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:32.759778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:32.760437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.760718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.760991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.772007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:32.773467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9790351c-a0a4-4413-91a8-281d1728fbc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:32.795948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:32.796925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:32.797758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:32.797758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:32.797758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:32.797758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 239/460] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:32.808412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:32.833658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04031a1-5d7f-498e-9967-2a19dfb99870 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.833933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04031a1-5d7f-498e-9967-2a19dfb99870 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.834434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04031a1-5d7f-498e-9967-2a19dfb99870 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.836201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 240/461] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:32.841242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cc9c-8d3a-45e0-ad18-584d70dfda27 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.841721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cc9c-8d3a-45e0-ad18-584d70dfda27 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.842188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cc9c-8d3a-45e0-ad18-584d70dfda27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.862002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-47f1cc9c-8d3a-45e0-ad18-584d70dfda27 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:32.863482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-47f1cc9c-8d3a-45e0-ad18-584d70dfda27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:32.933272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 241/462] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:32.943193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c17-de15-4168-944a-0598e3e0b616 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:32.943193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c17-de15-4168-944a-0598e3e0b616 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:32.943193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c17-de15-4168-944a-0598e3e0b616 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:32.943193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 242/463] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:32.943193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945c17-de15-4168-944a-0598e3e0b616 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:32.955569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1004ec-4d49-484d-b04a-ff2a444d745e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.000419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.001227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.001626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.002012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.002926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:33.003226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.005132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6489f8-728d-4a4a-ad68-c4fa78f7351c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.030871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 222/464] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:33.030871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.030871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.030871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.051241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.056001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:33.057206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:33.087250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945c17-de15-4168-944a-0598e3e0b616 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.089254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.131496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 223/465] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:33.133166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.149918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.150920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.151406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.151906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.169981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.172693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:33.188335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:33.189271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:33.190156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:33.190156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.190156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:33.192809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 243/466] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:33.201328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.289990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038104d7-ba19-4a33-a41a-b5aa7b9383f5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.328971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6a6e-8e28-45b3-a0c7-2d4ecc657daf None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.329569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.330541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.331137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.331474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.332245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.334329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.359084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.368223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.368501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.368771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.369417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:33.369722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.371167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.388340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 244/467] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:33.388340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.450867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 224/468] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:33 2022] POST /identity/v3/users => generated 246 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:33.452069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.492068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.500512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf681-24bc-4440-9d95-367572988da7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.541678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.542778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.543257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.543815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.556670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.558815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:33.584405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:33.585396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:33.586221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:33.586221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.586221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:33.586221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 245/469] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:33.587074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.592080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d97a-bf5e-4864-8a61-b080f5bf7ce3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.634589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.634589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.634908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.635160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.635653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.637184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.685445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.744597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e6b5-f097-4d3d-824e-1378308c7de5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.779900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 225/470] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:33 2022] POST /identity/v3/users => generated 248 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:33.779900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.790100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.790753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.791035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.791310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.805765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.807177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-af65d862-ba6b-42fc-8ac7-26414dbd1c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:33.824554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:33.825505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:33.826358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:33.826358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:33.826358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:33.826358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 246/471] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:33.827021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4298-6598-461a-bec8-baf13e00cc01 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.827306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4298-6598-461a-bec8-baf13e00cc01 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.830408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4298-6598-461a-bec8-baf13e00cc01 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.830408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 247/472] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:33.831494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4298-6598-461a-bec8-baf13e00cc01 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:33.903032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.927904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dafdfa-b312-44e2-9769-508ed6eca7a9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.962721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:33.969750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:33.970800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.971073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.971337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:33.972038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:33.972344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:33.972649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4298-6598-461a-bec8-baf13e00cc01 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:33.974503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27c19-e9cf-4931-92b7-104bd3ab6a84 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:33.989523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 226/473] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:33.990995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:33.991447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:33.991734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:34.015794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:34.024448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.025349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.025772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.026192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.026994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:34.027453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:34.029048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:34.042504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 248/474] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:34.043329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:34.095242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 227/475] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:34.096286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:34.179894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:34.191167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc035e0-e9c7-4c2a-b971-71b08603d119 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:34.231584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.232267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.232541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.232808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.233453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:34.234882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:34.252233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fa08e-d937-4f31-a8c8-96f97c2b77a5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:34.293804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.294539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.294817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.295126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.295837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:34.297353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:34.366303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 249/476] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:34 2022] POST /identity/v3/users => generated 249 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:34.417646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 228/477] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:34 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:34.439947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:34.455073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:34.563724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74e3f-bb3d-4be8-b567-f6044ced1030 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:34.603026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.603655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.603993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.604273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.604958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:34.606350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:34.645759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.sql.core [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:58:34.645759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:58:34.645759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [parameters: {'id': 'fe62c5f144cb440c8f9183d566dcfd6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:58:34.645759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81890) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application result.read() Aug 29 18:58:34.662207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:58:34.663125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application yield resource Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise self.value Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application session.commit() Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:58:34.664907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:58:34.666942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:58:34.669127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application result.read() Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:58:34.670697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [parameters: {'id': 'fe62c5f144cb440c8f9183d566dcfd6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:34.671720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:58:34.673719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:34.676184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 250/478] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Aug 29 18:58:34 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:34.676349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:34.778763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad88a-82c2-4b15-bad3-bc696203c563 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:34.808610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:34.817533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.818221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.818554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.818835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:34.830466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f03ef135-43a7-4102-acc3-26eb56ba9979 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:34.835446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:34.837550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:34.839540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 229/479] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:58:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:34.848016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:34.928557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:34.960788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dcbe-defc-4e7f-8e77-177676652289 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:34.998311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:34.998927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:34.999206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:34.999477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:35.000203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:35.000503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:35.001953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391788fd-b80c-4938-872b-e6916514ccdc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:35.010774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 251/480] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:35.638741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129917d-1b1b-4ed8-b25d-faac206772c1 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:35.639456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129917d-1b1b-4ed8-b25d-faac206772c1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:35.639456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129917d-1b1b-4ed8-b25d-faac206772c1 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:35.640996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 230/481] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:35.647028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239053e-43a9-4d10-b51c-2b9a46aa7e6a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:35.647549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239053e-43a9-4d10-b51c-2b9a46aa7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:35.647733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239053e-43a9-4d10-b51c-2b9a46aa7e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:35.669316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d239053e-43a9-4d10-b51c-2b9a46aa7e6a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:35.670568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d239053e-43a9-4d10-b51c-2b9a46aa7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:35.743483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 252/482] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:36.054592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129917d-1b1b-4ed8-b25d-faac206772c1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.167063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129917d-1b1b-4ed8-b25d-faac206772c1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:36.202000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:36.202632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.202911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.203238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.213733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:36.217202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226963-1520-4f4d-81a1-56c48530e265 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.217486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226963-1520-4f4d-81a1-56c48530e265 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.217762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226963-1520-4f4d-81a1-56c48530e265 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:36.219150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:36.221311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:36.224160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:36.225582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 253/483] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:36.225717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 231/484] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:36.228614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.228614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.228614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.229062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.272994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:36.274116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:36.278034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:36.340467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ddd6f-0797-4efb-acbb-c260f8541d3e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:36.377330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:36.377932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.378211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.378484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.379145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:36.380105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:36.381672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:36.390343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 232/485] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:36.396151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.489900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 254/486] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:36.490508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:36.553097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723911e-dc1d-4b2a-a949-349f5b1588a6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:36.560177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:36.591569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:36.592286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.592566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.592839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.607591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:36.609104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-64c9e72e-f30f-44e3-8e4a-d722077e140a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:36.632528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:36.633497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:36.634350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:36.634350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:36.634350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:36.634350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 233/487] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:36.639683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.747101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76112-7c26-4965-8f37-95fe22054f52 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:36.783165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:36.783777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.784082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.784357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.785009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:36.785317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:36.786728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be504-238a-4da4-bc32-b7fbaf9722ca None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:36.793835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bb9f-ad99-4dcb-b202-4de7b449f0e7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.794297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bb9f-ad99-4dcb-b202-4de7b449f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.794510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66bb9f-ad99-4dcb-b202-4de7b449f0e7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.798442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 234/488] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:36.799242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66bb9f-ad99-4dcb-b202-4de7b449f0e7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.800330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 255/489] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:36.803929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.804163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.804434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.823712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:36.825033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:36.881407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:36.908086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 256/490] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:36.909045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:36.929705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66bb9f-ad99-4dcb-b202-4de7b449f0e7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:36.969020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:36.969658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:36.969936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:36.970209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:36.999572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:37.011080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:37.012002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:37.012793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 235/491] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:37.021317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.056891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.058018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1f835-21ae-45af-9f18-33e3903e06d9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.095560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.096400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.096675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.096946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.110372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.111827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:37.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:37.142891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:37.145765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 257/492] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:58:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:37.151880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.161719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3030ca-23b8-4849-9291-5c255d5893a7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.201381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.201994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.202275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.202552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.203216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:37.203519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.205067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.214549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 236/493] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:37.219816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.233141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.292025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9377-85a5-4ad8-8865-60eeeb51bd3b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.315579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.329978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.331051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.331609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.332056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.346837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:37.346837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.346837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1741045-0698-40dd-9a72-6cf355258b18 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.347115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf020f-ff1d-4a33-87ec-a4865278d5bb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.375495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.376158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.376440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.376707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.377526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 258/494] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:37.378798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3102fa-1dac-4d94-b391-906f1d59dcfd None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.379087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3102fa-1dac-4d94-b391-906f1d59dcfd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.379419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3102fa-1dac-4d94-b391-906f1d59dcfd None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.381393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 259/495] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:37.385994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3102fa-1dac-4d94-b391-906f1d59dcfd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.392409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.393876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:37.409547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:37.410456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:37.412054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:37.412054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:37.412054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:37.415974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 237/496] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:37.416827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.498602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3102fa-1dac-4d94-b391-906f1d59dcfd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.500380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.519655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033e1d-7628-4686-b1bd-77d136ee2a55 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.537323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.538070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.538345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.538615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.549586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:37.549873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.554913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-571e85cd-ce4e-47e9-9d23-ed41a4b56b94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:37.555193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:37.556271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:37.557222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:37.557817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.557817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.557817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.561252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 260/497] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:37.562292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.562569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.562838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.582116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:37.583895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:37.637332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.638833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.659080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 261/498] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:37.659958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.738486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 238/499] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:37.739704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.767432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.769974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:37.778317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477f0e6-8714-4018-842b-10166f916b59 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.813375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.813999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.814282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.814555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.815537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:37.815895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.818325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377f71-7895-4169-ba4d-67eec45d46ae None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.831989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 262/500] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:37.834121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6742-100c-4061-9bb3-fa5a85f6fee2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.834337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6742-100c-4061-9bb3-fa5a85f6fee2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.834612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6742-100c-4061-9bb3-fa5a85f6fee2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.840034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 263/501] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:37.840034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c6742-100c-4061-9bb3-fa5a85f6fee2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.847033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726916bd-14bb-4bf3-859d-54669f267ebe None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.884854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.885458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.885734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.886004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.886652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:37.886953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:37.888365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:37.900667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 239/502] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:37.900667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:37.952574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c6742-100c-4061-9bb3-fa5a85f6fee2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:37.987307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:37.988241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:37.988526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:37.988842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:37.995916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.001342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.002289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca6ded-5d7b-4b99-b853-f2545bdd06d6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.002849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-023fd5a0-482f-47df-baea-58606c5ec4dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:38.017579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.018522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:38.019357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:38.019357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.019357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:38.019357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 264/503] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.019795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.020751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.020853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.028312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.038295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.042170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.042170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.042170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.042426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:38.042426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:38.046392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.047884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:38.060939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.062026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 240/504] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.063104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.111082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 265/505] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:38.111844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.173364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e33eb-51d9-4484-81e9-984c6e1f73cb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.205860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.206471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.206742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.207006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.217699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.219181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.221966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ced6e-7ae9-49c8-9d1c-15562a453e4d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:38.234271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.235184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:38.236073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:38.236073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.236073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:38.236073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 241/506] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.236299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.245453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.266276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.267173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.267449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.267717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.269927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.280079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:38.284827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.285819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:38.286658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 266/507] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.287388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.351348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b8604-f8d8-4fc9-9d5e-e5bd77b7894c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.386206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.386726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.386998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.387263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.387915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:38.388213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.389565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.393880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e4be5-1fd8-42f1-8c7f-eb17bdb8b938 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.401782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 242/508] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:38.402577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.431183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.431774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.432071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.432338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.433052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:38.433350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.434773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.452560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 267/509] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:38.453354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.471154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.505399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ff475-b266-430d-9bde-e1dcc44f8ed8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.509233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.538627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.539228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.539507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.539779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.540451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:38.540804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.542192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.557484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 243/510] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:38.557484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.557705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d849a-2e6e-4f58-b345-570938376e6c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.591026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.591633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.591964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.592236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.592890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:38.593186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.594555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.602550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 268/511] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:38.603318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.661374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8743f-74f0-47ff-8ff1-11124d3b0496 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.685558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.694475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.699056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.699056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.699056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.702765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1e293-0491-4c85-a289-10a3486cb260 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.705664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.707111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.716131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:38.721334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.722219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:38.728306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:38.728306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:38.728306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:38.728306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 244/512] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.728306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.736192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.736810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.737082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.737353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.744609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.746986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:38.759669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.761335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 269/513] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.763056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.829285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846fcf-7c0e-49b9-950c-65561fa1b2e1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.860128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40751b-a1f9-4ba3-9970-9933d4652322 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:38.862360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.863001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.863259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.863528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.896887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:38.897506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:38.897788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:38.898055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:38.905424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.906863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.909620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:38.917930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:38.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 270/514] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:38.923935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:38.932203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:38.933659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:38.956238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.025024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abb47-e00d-482a-8803-c2773e28dc31 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.041099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 245/515] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:39.041852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.059643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.060303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.060571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.060839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.071623 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.073110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:39.088015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:39.089100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:39.089921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:39.089921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:39.089921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:39.089921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 271/516] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:39.089921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.121136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.141902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7378-725e-4f40-a7da-f61d7d032ee9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.174718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.176584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.177187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.177471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.178078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.178732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.180479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.188895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d6b96-6d0d-4f4c-ba83-74ccb202fffb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.191401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 246/517] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.207349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.224744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.225373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.225652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.225926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.226641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.226946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.228482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.246889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 272/518] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.247691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.291333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ce2c-f410-4201-a660-306e37ecba71 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.324894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.325519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.325794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.326066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.326787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.327086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.328572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.347421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7b63-65b0-43e3-b6b6-30e6799b029e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.348013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.368153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 247/519] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.369030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.387421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.388176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.388454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.388724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.399092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.400579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.407289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:39.416197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:39.417058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:39.417889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:39.417889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:39.417889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:39.417889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 273/520] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:39.418105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.496334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861699-804d-485e-9e3c-6787af6ec7cd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.520395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7ad-5365-467b-a32b-2ff2340552c3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.529914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.530513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.530790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.531062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.531698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.532036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.533440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.544759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 248/521] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.545563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.553405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.554008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.554290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.554560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.555196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.556686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.601160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.644239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce1cb8-0f2f-4c57-b86f-ac8ebb007cb2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.661584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 274/522] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:39 2022] POST /identity/v3/users => generated 247 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:39.662586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.688937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.690001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.690895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.691343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.693642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.694119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.696643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.706077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 249/523] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.707382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:39.744407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.802295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b858d2d-235a-435c-9e1d-02cdcab64c12 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.852427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.853568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.853568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.853568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.894396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a27f-911e-4637-bdfe-f3c462f769ed None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:39.918029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:39.939759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.941271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.941509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:39.942482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:39.942960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:39.943587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:39.945394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:39.945909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:39.947626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:39.964062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 250/524] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:39.965608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.020353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.066469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 275/525] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:40.068275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.099427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc7f7-6d5d-4088-946f-ff6422634c86 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.145969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.146599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.146870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.147137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.155101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.156652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:40.169686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:40.170619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:40.172683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 251/526] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:40.177169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.184820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e59bb-97a5-461c-8163-f2715a9a1a11 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.221192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.221836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.222165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.222446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.225938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.230490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.232128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:40.245172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:40.247225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 276/527] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:40.249014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.268035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.290320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537ed6d-e571-4377-8c59-3cdab216c845 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.330174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.331058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.331349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.331624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.343780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.345580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:40.362643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:40.365388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:40.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:40.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:40.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:40.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 252/528] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:40.367569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.368228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b3ed-8504-4ad8-b653-81ffb906e123 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.403349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.404371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.404893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.405524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.406409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:40.406876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.408502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.429399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 277/529] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:40.430458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.478472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.488460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f27ce8-a529-4dc0-aaae-de4d07c6b0cc None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.509896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.527464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.530739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.531033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.531309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.532038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:40.532342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.533793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.543314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e7707-2570-4295-82ec-207fb23eb3e4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.551826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 253/530] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:40.552779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.582938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.583469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.583748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.584085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.584731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:40.585210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.586709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.604190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 278/531] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:40.604190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.685408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edd927-9ee2-4d60-a6ae-93880af3529e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.721763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.722749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.724112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.724622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.734279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b223c-d861-411b-9963-3f3467496f06 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:40.749349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.776262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:40.780763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:40.781390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:40.781670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:40.781944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:40.782955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=fa271ae95240497e9be3175e44a2b545) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:40.783267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.785509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.799880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 279/532] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/role_assignments?user.id=fa271ae95240497e9be3175e44a2b545&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:40.799880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:40.836916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:40.839223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:40.988409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027316b-9286-4e9c-be26-f55e6998f8b0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.029839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.030254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.030534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.030806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.037861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.039304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:41.052849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:41.054922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:41.055808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 280/533] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:41.062235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.128313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:41.154809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 254/534] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:41.160168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.171048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d03c6-94c2-4def-899b-a6ee68caa267 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.234674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.235287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.235564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.235836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.236553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:41.236852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.244492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.273641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 281/535] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:41.280288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:41.282947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.343242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5d29-71cc-41ff-a020-0155e4f6dcc3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.378113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.378800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.379022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.379812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.391224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.392723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:41.412981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:41.413893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:41.413893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:41.413893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:41.413893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 255/536] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:41.417100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.491675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e1a3-9c2d-4858-b9bb-bf2e594d2a51 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.528268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:41.541449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.542249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.542665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.543144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.544031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=94f1f1b96cd6477581e1040d8d8e0529) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:41.544495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.546057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.556167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 282/537] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/role_assignments?user.id=94f1f1b96cd6477581e1040d8d8e0529&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:41.588419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb56b-0505-4577-a973-145f8165037b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.596063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:41.643994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.644530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.645338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.645776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.646441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:41.646746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.648185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444b2c1-51e2-4488-8268-0d2cda4b93df None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.671528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 256/538] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:41.677320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.801708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a5d1-dfa9-458e-b8f1-0731dae12840 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:41.837228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:41.838199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.838686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.839191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.846531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d505193-1801-41ed-981d-ebeabc7fe91d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.846820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d505193-1801-41ed-981d-ebeabc7fe91d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.847058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:41.847173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d505193-1801-41ed-981d-ebeabc7fe91d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.848975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 257/539] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:41.849283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:41.854815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991d1d-b568-4382-bc34-25eb79bd8685 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:41.855681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991d1d-b568-4382-bc34-25eb79bd8685 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:41.856210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991d1d-b568-4382-bc34-25eb79bd8685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:41.867054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:41.867972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:41.868791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 283/540] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:41.877392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:41.885202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1a991d1d-b568-4382-bc34-25eb79bd8685 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:41.886677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1a991d1d-b568-4382-bc34-25eb79bd8685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:41.961296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:42.009461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 258/541] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:42.011410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65bc0-2613-46ac-a5c4-d4be510b34bb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.011694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65bc0-2613-46ac-a5c4-d4be510b34bb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.012037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65bc0-2613-46ac-a5c4-d4be510b34bb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.017755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 259/542] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:42.020608 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.020925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.021211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.044280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:42.045805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:42.058334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb9cb8-a13a-429a-a2de-51856e03a625 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:42.135755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:42.136393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.136671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.136995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.137698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:42.138000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:42.139520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:42.164406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 260/543] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:42.165873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 284/544] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:42.171793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.211566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:42.315263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c172-9a91-4655-a2cc-b9d40a447178 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:42.344448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.359221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:42.359834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.360140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.360409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.361134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bb59243400b44b8e998daaa4bd8f1ef0) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:42.361439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:42.362896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:42.369209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 261/545] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/role_assignments?user.id=bb59243400b44b8e998daaa4bd8f1ef0&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:42.493971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:42.497243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fdedc-e20a-4ff3-84cb-f1cd25e682f7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:42.535128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:42.536051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.536507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.536936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.548875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:42.554448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:42.556524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:42.558544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 285/546] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:42.561673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.607840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.704228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:42.723633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9561d-f6db-49ee-adf0-abbb82863760 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:42.785822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6f537-1842-4be4-8c9f-0c3e8dc56d66 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:42.789234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:42.789846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.790120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.790386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.791045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:42.791342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:42.792730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:42.812072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 262/547] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:42.820010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:42.851815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:42.852113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:42.852386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:42.871611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:42.877760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:42.878692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:42.880125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:42.881544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 286/548] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:42.887322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:42.923404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:42.983938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f499-8c1b-4809-950a-ff4737262bd8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.019978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddca1-b532-495c-9fe9-709919174192 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.030823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.031586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.031990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.032358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.047432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.049273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.064972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.066167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.066642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.067059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.067893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:43.068338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4c185379-ca2e-434c-9431-b97893cc4b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:43.068862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:43.069780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:43.070631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:43.070631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:43.070631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:43.070631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 263/549] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:43.072320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244466-9428-4cd1-9b97-f08ea31b9b9a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.072656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeacb4b-0d52-4f47-a3ec-5084f34ab7ca None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.072759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244466-9428-4cd1-9b97-f08ea31b9b9a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.073029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244466-9428-4cd1-9b97-f08ea31b9b9a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.078361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 264/550] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:43.078361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be244466-9428-4cd1-9b97-f08ea31b9b9a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.079645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.089742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 287/551] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:43.095011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.095011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.095011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.120156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:43.121677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:43.196094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be244466-9428-4cd1-9b97-f08ea31b9b9a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.207877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 288/552] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:43.207877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.226048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.263189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.267236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.267574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.267917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.268565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:43.268866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.270845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.298286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 265/553] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:43.302331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.382953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a2d40-3ff8-4aa6-854b-c2df0a5b0dcb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.427968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.429375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.429992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.430487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.439913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.443420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.445263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:43.462362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:43.463309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:43.464285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:43.464285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:43.464285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:43.464285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 289/554] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:43.470168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.535385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295907-e54a-478f-8ff5-3e82d277c9db None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.545615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.569838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.570380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.570615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.570882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.578592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.580079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:43.593711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:43.594712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:43.595632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 266/555] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:43.596230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.598948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c000-91ec-4b22-93be-1c51cee6d023 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.650428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.651727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.652362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.652837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.654147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:43.654792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.656701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.678202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 290/556] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:43.679395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.708834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1907f7-f9f4-45a0-982c-6c3c7dcabdf1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.736237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.744122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.744844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.745181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.745652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.756302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:43.761257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:43.762698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:43.763549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 267/557] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:43.764264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.810885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488546f1-22fb-4d46-b329-59739f5000e7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.837066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:43.849848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.850589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.850956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.851404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.852331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:43.853025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.854328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.875210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 291/558] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:43.876396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.896918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf403a-3b55-4aa1-8385-cc380a666394 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:43.943152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:43.944636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:43.945068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:43.945765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:43.956320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:43.958635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:43.978550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:43.979804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:43.980953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:43.982815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 268/559] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:43.983743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:43.992924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64761c31-0dd3-43f8-887f-7e062db98509 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.030908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.031799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.032613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.033035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.033643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.034453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:44.034453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.036524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74317b9-be16-415c-b666-50aae9ecb5ed None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.051667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 292/560] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:44.051667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146141b-ddec-40fd-9203-07a66d885813 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.051667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146141b-ddec-40fd-9203-07a66d885813 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.051667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146141b-ddec-40fd-9203-07a66d885813 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.052621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 293/561] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:44.053427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146141b-ddec-40fd-9203-07a66d885813 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.105294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf49e-b490-40e4-9964-3f776b70b329 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.107536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.147738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.148381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.148661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.148931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.172467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146141b-ddec-40fd-9203-07a66d885813 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.216363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.217395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.217885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.219133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:44.219647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.221108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.223333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.225112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f884b8-462d-4c1c-974a-08e4f36503fc None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.241605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 294/562] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:44.247010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.279671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.344191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 269/563] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:44.345813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.346297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.346772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.364318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb03f-61c0-48da-8a39-235a85b0b4b7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.365572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:44.366198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.366771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:44.425125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.427001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.427509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.428099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.446116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.447746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.463931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 270/564] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:44.463931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:44.464204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:44.465138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:44.466042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:44.466042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.466042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:44.470268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 295/565] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:44.471583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.591315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89410b23-b7b4-4d74-b02c-0f9a0959ff00 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.593092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.624801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce600-e3fb-4f93-9915-3b6ff80dd6db None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.628266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.629037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.629322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.629594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.640767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.642246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:44.657505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:44.658713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:44.659801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:44.659801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.659801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:44.663988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 296/566] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:44.664106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.664732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.664853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.665007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.665274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.674172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.678024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.679488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-04a796f6-4600-4ecc-9ae0-8f1459537917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:44.695562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:44.696750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:44.697864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:44.697864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:44.697864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:44.701298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 271/567] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:44.702627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66e03-de57-49c3-af86-e798371ba6a7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.702912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66e03-de57-49c3-af86-e798371ba6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.703187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66e03-de57-49c3-af86-e798371ba6a7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.709296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 272/568] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:44.709296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66e03-de57-49c3-af86-e798371ba6a7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.780810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa441f-7a41-48ad-ace5-66514cb61e57 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.816163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.817451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.817849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.818087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66e03-de57-49c3-af86-e798371ba6a7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.818187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.818802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:44.819332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.821400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.835252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 297/569] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:44.835252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.848160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.859100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.859802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.860108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.860380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.861026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:44.861328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.862712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a4ea5-b6b4-479d-8fe2-bf768c2dbb7b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:44.875278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 273/570] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:44.876482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.877048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.877225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.901615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:44.902814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:44.906345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:44.943052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5dfb-c1c1-4129-8067-9181b25aca26 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:44.979059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 274/571] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:44.979949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:44.982783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:44.983394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:44.983696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:44.983994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:44.984637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:44.984936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:44.989153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.003317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 298/572] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:45.004332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.102584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d24bc-f76b-4a96-a608-ec89dc9ec1e0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.118010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.131003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c94837-e2b3-4398-8714-730de8e4b80f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.138337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.138964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.139235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.139504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.149526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:45.154624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.155634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:45.156529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 275/573] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.157213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.164113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.173119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.178469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.178469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.178469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.183601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.185273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:45.197585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.198542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:45.199469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:45.205919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 299/574] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.205919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.271111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67840-b56b-4780-aa57-a76a6eb96a82 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.307566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62e37-13c6-4b0c-bd49-db294159694c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.309720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.310357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.310634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.310904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.317555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.319029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:45.330752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.331630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 276/575] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.332696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.344827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.345455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.345729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.345996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.353574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.355626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:45.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.369959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:45.370843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 300/576] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.371317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.376130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.419435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.442924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8db17c-e3cd-41a8-94a2-2f1e5cea26a5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.481977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.482578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.482853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.483123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.483781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:45.484104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.485580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.494706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 277/577] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:45.495532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.527951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976971e-455c-46f8-bd07-4928a3919081 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.564646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.565261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.565540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.565813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.566546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:45.566872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.568406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.586393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 301/578] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:45.587220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.600448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.606160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce47215-b9c2-4c79-80c5-feace4d225ac None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.641002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.641634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.641911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.642185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.653947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:45.658814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.659824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:45.661395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 278/579] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.662082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.699277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5491e-2ce4-44f5-968d-bc6638b8fa4a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.701537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.734679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.735283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.735558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.735830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.736545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:45.736849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.738281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.749403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 302/580] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:45.750187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.772021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f453e-5672-4111-a63a-735bc693ca02 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.805377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.805972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.806246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.806516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.807199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:45.807495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.808972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.820974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 279/581] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:45.821770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.840251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.859719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06e2bd-1f1a-4ae4-873d-b387684c8a43 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.897273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.897897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.898171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.898441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.908723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.910185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:45.924244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:45.930170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:45.931326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:45.931326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:45.931326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:45.931326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 303/582] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:45.931326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:45.936117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df0aa-bc5f-40a7-aaab-45beea7b27ac None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:45.947226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:45.969889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:45.970448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:45.970720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:45.970996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:45.986211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:45.988462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:46.005270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:46.006267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:46.007530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:46.007530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:46.007530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:46.007530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 280/583] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:46.007530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.034599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f71be1-6883-4efd-8b28-3046de31f8c8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.068767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.069378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.069657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.069929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.070590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.070895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.072299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.079136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.080384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 304/584] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.086176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.114629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4c496-615e-4b2b-a1d3-f05e8a95fe02 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.148660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.149262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.149540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.149811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.150527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=d045b7cdd6024b6b82f3c4864c01e7d3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.150834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.152327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.163913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 281/585] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/role_assignments?user.id=d045b7cdd6024b6b82f3c4864c01e7d3&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.163913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.169284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.195110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5171-9567-4a5d-8775-4eb3f2b3e239 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.234972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.235683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.235988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.236281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.254636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=bf0af6b468c7442e818efec1543b7eef, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.256384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.270692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7267-6e71-48dd-a7b0-420cb2e3beff None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.303463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.308599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.309210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.309489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.309759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.310402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.310703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.312195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.325654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 282/586] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.326499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.327017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 305/587] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:46 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:46.328435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.451437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe893d-1ddd-4f48-bbbb-5f0cec184438 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.510200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.518388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.519164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.519593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.520029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.522039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d759-c1d7-4635-bfa5-663d3c246557 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.532820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.534716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:46.555162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:46.556486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:46.557595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:46.557595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:46.557595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:46.559350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 306/588] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:46.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.566109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.566851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.567242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.567594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.568435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.570422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.583560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 283/589] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.584599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.649196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.680141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ef14-ca3b-4c39-a0c3-522775f85c0b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.705774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f23d8-acd3-4b24-b161-4254cc73fd0a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.716390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.717119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.717469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.717821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.726176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.727710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:46.739638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:46.740834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:46.742013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:46.747743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 307/590] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:46.748681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.748928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.749707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.750130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.750548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.751333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.751925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.753562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.768556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 284/591] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.770110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.798507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.860998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55fec0-fd08-45d2-b548-eb69eb1404f6 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.897434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6052b48-fbb9-46e1-815d-3e730718772b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:46.905240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.905885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.906163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.906434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.913603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.915147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.920240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:46.927663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:46.928817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:46.929906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:46.932675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 308/592] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:46.933348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:46.937005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:46.937718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:46.938064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:46.938397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:46.939186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:46.939555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:46.941099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:46.959954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 285/593] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:46.961038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.046377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.050137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78e14f-f5f0-4341-a9c9-a66ac15856cb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.067307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413f1a-2653-47c9-9f0a-5462f7ab03bd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.086830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.087705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.088209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.088645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.097702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.100397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.103319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.104067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.104434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.104796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.105574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:47.105973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.107505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:47.113596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:47.114720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:47.115880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:47.118000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 309/594] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:47.119002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.128016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 286/595] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:47.129246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.160790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.226507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559fca8-117c-4877-9e00-ce351b671968 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.231632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f856eb-0d80-44d4-87d8-2e9e5beb0161 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.263135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.264272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.264843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.265366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.266300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.267041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.267413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.267774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.268649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:47.269049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.271310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.280444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.282304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.291309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 287/596] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:47.292312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:47.299013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:47.299966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:47.300941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:47.300941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:47.300941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:47.302815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 310/597] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:47.303783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.324484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.406216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.419563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0faf5-6156-452c-b522-8c98ebcf5c90 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.446030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1471392-53e4-4eec-ad35-17335536b584 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.458378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.459373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.459870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.460377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/fa271ae95240497e9be3175e44a2b545/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.490926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.491564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.491865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.492160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.511598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=fa271ae95240497e9be3175e44a2b545, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.512431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.513265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.514094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:47.530191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:47.533641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:47.535981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:47.535981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:47.535981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:47.538304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 311/598] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:47.539356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.610897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.620290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 288/599] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:47 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/fa271ae95240497e9be3175e44a2b545/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:47.622463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.719541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dce8f-d47b-45dd-9325-8026dac4183e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.804447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.805066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.805339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.805606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.806250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:47.806545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.807963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.825724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 312/600] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:47.830881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:47.865941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189457-60e0-4dc7-98e1-b1c6884c9db6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:47.919323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:47.920527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:47.922125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:47.922634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:47.923677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:47.924815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:47.926762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:47.938118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:47.950581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 289/601] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:47.959476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.026119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ebb98-7355-4f79-8e88-a6e6b6d19676 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.084541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc303fd-7810-412d-a418-6174c3303c2c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.088225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:48.116761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.117361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.117647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.117923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.134419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.135066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.135343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.135762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.137536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.140143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.152154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.153646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-13df3e36-559b-448f-b0ae-005ac8104cbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:48.171465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:48.172430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:48.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:48.173704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:48.174460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:48.175314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:48.175755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 313/602] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:48.175822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 290/603] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:48.175908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48601d-193c-4c5e-9e0e-d62abd312a49 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.175908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48601d-193c-4c5e-9e0e-d62abd312a49 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.175908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48601d-193c-4c5e-9e0e-d62abd312a49 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.181815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 314/604] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:48.182071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.187148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.187431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.187706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.210788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:48.212344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:48.261692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:48.291800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 315/605] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:58:48.293461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.318010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f3863-9cd4-4e03-a9c1-f1fb61aaef76 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.412038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa61be-e9da-4540-a701-f27302f084f1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.450987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:48.457819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.458825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.459867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.460098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.460398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.460670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.461334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.461612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.462303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:48.462874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:48.463178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.464886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.466958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.468840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.497107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 316/606] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:48.497408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 291/607] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:48.503721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.508237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.656562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41df4e-cae6-4dcf-bd4f-2de2c8366fd5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.664278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d03b6-096c-44a4-b5d8-db4a7af213c0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:48.682514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:48.695389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.697083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.697083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.697239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bb59243400b44b8e998daaa4bd8f1ef0/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.700833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:48.701495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.701770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.702046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/94f1f1b96cd6477581e1040d8d8e0529/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.716711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=bb59243400b44b8e998daaa4bd8f1ef0, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.717018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=94f1f1b96cd6477581e1040d8d8e0529, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:48.718526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7804c-9502-4c20-95d3-75cab1d81128 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.718837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb7e55-edf5-4199-aa27-5d07816e69c1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:48.822403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 317/608] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:48 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bb59243400b44b8e998daaa4bd8f1ef0/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:48.823984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491b73-3f0b-40fc-af23-0ea4edeb757f None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.824687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491b73-3f0b-40fc-af23-0ea4edeb757f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.826404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491b73-3f0b-40fc-af23-0ea4edeb757f None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.827973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 318/609] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:48.828649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491b73-3f0b-40fc-af23-0ea4edeb757f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:48.852627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 292/610] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:48 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/94f1f1b96cd6477581e1040d8d8e0529/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:48.858719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:48.858719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:48.858719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:48.893811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:48.895241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:48.897528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:49.023101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 293/611] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:49.047131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491b73-3f0b-40fc-af23-0ea4edeb757f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.087243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.087939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.088239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.088485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.100740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:49.104947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:49.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:49.106712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 319/612] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:49.112183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.218703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:49.223364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3e5e2-1a5b-4615-ad51-fa9a469831f9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.261524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.262048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.262328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.262582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.263248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:49.263551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:49.264993 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:49.274287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 294/613] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:49.279797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.325071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.414162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:49.445901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dffdd8-685e-45dc-a04d-1a6e729598f7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.498355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dfd5-fa7a-41ff-ad83-93ded5452e07 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.513826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.514826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.515309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.515791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.530811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:49.533739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:49.560739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:49.561730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:49.562584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:49.562584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:49.562584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:49.567967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.568846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.569315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.569791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.573147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 320/614] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:49.578505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.591513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:49.597097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:49.598522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:49.599601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:49.605066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 295/615] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:49.624878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.650956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:49.737345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ee08-ea0d-4e84-b040-3a2ecfb2a20f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.784456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cfd10-d412-4ef0-ada6-f01c8f1f9c13 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.785473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.786389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.786924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.787388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.788283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:49.788776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:49.790354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:49.807676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 296/616] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:49.824361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.835442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.836103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.836384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.836658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.837318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:49.837813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:49.838983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af57f-38fe-483b-9e2a-d28a32ab65f7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:49.841688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:49.878574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 321/617] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:49.882320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d558c-1fdb-481d-b10e-addbc5e73e4a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.882606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d558c-1fdb-481d-b10e-addbc5e73e4a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.882878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d558c-1fdb-481d-b10e-addbc5e73e4a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:49.888275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 322/618] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:58:49.889037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d558c-1fdb-481d-b10e-addbc5e73e4a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:49.914768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:49.935795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a76dda-a2e6-42a8-824a-0178a5f0cf24 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:49.994540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:49.995184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:49.995458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:49.995726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.008126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.009583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.022605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d558c-1fdb-481d-b10e-addbc5e73e4a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-6096f738-9ab7-4c9e-a4b1-e414af6ce28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:50.027563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:50.028513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:50.029612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:50.029612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:50.029612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:50.029612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 297/619] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:50.030570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.030855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.031125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.054102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:50.055507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:50.075155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.076220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.076829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.077301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.086143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.088008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:50.102268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:50.103207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:50.105295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:50.107987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 323/620] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:50.109180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.137312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 298/621] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:50.138449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.205764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.227307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.275523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641b37a-d771-4a70-afe6-6a29d756f8b9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.311344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094680b6-6881-4094-9384-3526d1bfa3c8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.320102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.320866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.321252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.321652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.322502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:50.322919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.327876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.346896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 299/622] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:50.353007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.388405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.390529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.390980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.391478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.392411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:50.394691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.397863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.421789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 324/623] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:50.428890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.507954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a2f69-7260-4c4d-8ddf-162648151312 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.534951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.548211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.550765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.550765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.550765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.563532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.565047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.575038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d911-3c1f-4edf-ac59-a8811b055985 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:50.584083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:50.585077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:50.586025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:50.586025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:50.586025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:50.590797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 300/624] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:50.591688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.603521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.629653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.632574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.633121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.633640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.648654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:50.655656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:50.656680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:50.657567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:50.660021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 325/625] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:50.662419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.733127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5ff1f-3e7d-4022-a41c-66585d0a7762 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.774373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b92a31-0d94-4909-9337-05ebc8612109 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.780254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.781067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.781495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.781940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.793734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.795451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:50.811834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:50.813000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:50.814215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:50.814891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:50.815599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:50.816013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:50.817030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:50.817639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 301/626] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:50.818784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.822389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:50.822838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:50.824555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:50.847143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 326/627] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:50.847143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:50.885792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.886311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:50.968956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220416-c0cf-467d-b7c4-1760bce241d0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:50.970622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749efdb1-3add-4df3-a14b-c86ff3bf35dd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.012132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.012994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.013421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.013857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.014635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.014890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:51.015334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.015531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.015950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.016330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.017163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:51.017528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.017657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.019253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.032232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 302/628] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:51.033368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.043546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 327/629] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:58:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:51.044566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.139959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.146906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e44f-f6aa-4c49-9e50-2df44b921cf7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.147765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.155012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd477-cffe-4dae-aef5-043e155d133b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.189813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.190710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.191037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.191924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.202735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.202735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.202735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.202735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.203061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.203061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.208747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.210184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:51.212489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:51.213551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:51.214662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:51.216169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 303/630] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:51.217220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-cf98d584-eea3-4f41-8827-4f0b05d1c22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:51.233853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:51.235114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:51.236266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:51.236266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:51.236266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:51.239995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 328/631] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:51.241956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c2cb-6910-4a01-9c6c-2cfea1382387 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.242260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c2cb-6910-4a01-9c6c-2cfea1382387 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.242534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c2cb-6910-4a01-9c6c-2cfea1382387 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.243895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 329/632] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:51.244590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c2cb-6910-4a01-9c6c-2cfea1382387 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.356273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c2cb-6910-4a01-9c6c-2cfea1382387 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.358757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30c60-2eaa-4420-9b9f-26de0c85af9f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.390277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.390818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.391096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.391367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.392109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:51.392415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.393735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.393856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.394355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.394660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.394935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.405691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 330/633] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:51.406543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.416626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.420061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.464480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.466026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc825e7-24c4-408b-9202-480cb64af107 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.512068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954e337-df79-43e4-97aa-567df1b38395 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.545980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.546576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.546859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.547127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.547818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:51.548224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.550143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cc943-7f89-466c-b676-8b88114d3560 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.563053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 331/634] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:51.564286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.564572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.564850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.570627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 304/635] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:51.571913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43b1-fc89-49a5-9656-94675775e3e6 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.572194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43b1-fc89-49a5-9656-94675775e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.572467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43b1-fc89-49a5-9656-94675775e3e6 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.573915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 305/636] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:51.574375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f43b1-fc89-49a5-9656-94675775e3e6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.589799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:51.590986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:51.638919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.656544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.663975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 332/637] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:51.664861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.692576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f43b1-fc89-49a5-9656-94675775e3e6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.730351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.731005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.731281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.731553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/d045b7cdd6024b6b82f3c4864c01e7d3/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.750065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=d045b7cdd6024b6b82f3c4864c01e7d3, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.752777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d00da-efd2-413b-ae4c-653eef876428 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.770769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c071159-eb7b-4eee-9ef3-c59336161b50 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.805254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.805882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.806151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.806417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.813508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:51.814942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:51.826566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:51.827712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 333/638] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:51.828746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:51.864478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 306/639] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:51 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/d045b7cdd6024b6b82f3c4864c01e7d3/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:51.867304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.867545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.867888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:51.888905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:51.906374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:51.908434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:51.944507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fec4c-08a8-4888-a117-d9e8a0a1aa28 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:51.990503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:51.991190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:51.991469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:51.991765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.006535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.008012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 307/640] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:52.008159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.009549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:52.025451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.026448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:52.027340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:52.027340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.027340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:52.027340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 334/641] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.028013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.056755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.130385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e888489-d10d-41b5-b902-7cdfe7f8422d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.168411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444609af-732e-490f-9636-bf50957b7e0a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.173468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.174271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.174697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.175111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.176167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:52.176776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.178429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.179470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.200171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 308/642] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:52.201417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.216640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.217604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.218063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.218608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.231544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:52.237713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.238928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:52.239767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:52.247452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 335/643] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.248681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.317035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.323705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91f2dd-a5f1-432d-b112-cd42b270a27f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.361221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56ba2-ee01-45c7-b1a1-2ab77576c290 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.364408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.364651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.364919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.365186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.365819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:52.366116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.367592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.381088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 309/644] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:52.381088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.402316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.402931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.403209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.403474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.441084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.476362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.477810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.494960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602cf1-3ce4-4418-81ae-e59d013c2882 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.531186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.531720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.532021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.532289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.532939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:52.533239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.534845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.543143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 310/645] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:52.543971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.562927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.588156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 336/646] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:52.588942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.655730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcc7aa-1922-46a0-8458-f49f1bac28a6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.660319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.695202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f125d-9b32-41a6-b8df-ea425c62dc4d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.699009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.699640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.699948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.700254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.712356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:52.717098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.718084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 311/647] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.718959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.737373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.738273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.738577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.738858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.746914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.748398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:52.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.760864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:52.761751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 337/648] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.762232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.803592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.825684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56ce4-de13-4999-9ee8-3152e0aacc20 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.862023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.862682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.863508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.863830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.874184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.875692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.876694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2fa2-6285-4078-af16-eefe9d3e0937 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:52.893043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.894001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:52.895005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:52.895005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.895005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:52.895005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 312/649] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.895226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:52.911563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:52.921283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:52.922196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:52.936036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:52.936036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:52.937366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:52.938832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:52.955511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:52.956538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:52.957558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:52.957558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:52.957558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:52.957558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 338/650] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:52.957558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.014252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91ec54-4b34-4d7d-bee4-86de72aea345 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.042900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.051567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.052205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.052483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.052861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.053526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.053950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.055324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.069259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 313/651] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.070055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.073268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e4d12-3632-4c8a-b1f4-677112f4c510 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.107727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.108365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.108648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.108924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.109622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.109935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.111642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.129664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 339/652] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.129664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.170099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.184981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae557c-6c06-441f-923b-f0c4938e7e62 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.225146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.225614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.225892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.226165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.226801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.227166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.228578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.239008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107ccda-fddb-48fe-8faa-3e743e6058c7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.241976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 314/653] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.242328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.281207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.281851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.282131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.282405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.283054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.283359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.285283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.297213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 340/654] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.298031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.304153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.358558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a7b6-bdd2-413c-bcb9-b1518730555b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.397821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.398458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.398757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.399208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.405418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.408320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71468-fc38-4ae4-be5c-e5057ac163a4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.411605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.412844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:53.428801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:53.429967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:53.430834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:53.430834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:53.430834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:53.430834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 315/655] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:53.430834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.449080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.449690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.449963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.450240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.450966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.451265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.452750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.491988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 341/656] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.505311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.540998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f170cb-11bd-43e1-a0b0-743c46f4747a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.542997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.584469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.585173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.585450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.585775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.594529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.596117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:53.609514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:53.610509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:53.611429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 316/657] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:53.612051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.658579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875701a-4b16-45f8-8743-84ec675e7319 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.697083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.697724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.697996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.698265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.706065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.707575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:53.720775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:53.721696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:53.722653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 342/658] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:53.723190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.741385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03409df9-e472-43c1-803c-6d5b9ac27b82 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.758539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.779585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.780219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.780497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.780764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.781469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.781768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.783172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.795224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 317/659] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.796107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.826314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:53.861769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98921c07-8b33-414c-ae7e-2c0f82ce11ae None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.901714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.902331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.902608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.902877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.903740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.904161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.905598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.913396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e68c7c-16d0-410e-b51f-982c7cd58704 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:53.930190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 343/660] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.930190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:53.959422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:53.960324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.960698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.960896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.961752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:53.962362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:53.963812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c81a3-50dd-456a-9653-b65423ec24b1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:53.985243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 318/661] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:53.985243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d211-b4b2-4409-8d2c-facd6531afe5 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:53.985243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d211-b4b2-4409-8d2c-facd6531afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:53.985243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d211-b4b2-4409-8d2c-facd6531afe5 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:53.985243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 319/662] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:53.987269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641d211-b4b2-4409-8d2c-facd6531afe5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.033997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.060736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8dbe-4553-405e-bc46-ff77119193da None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.087109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.112044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641d211-b4b2-4409-8d2c-facd6531afe5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.122702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.123436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.123786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.124187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.131797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.133344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:54.146263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:54.147226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:54.148462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 344/663] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:54.149818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.158357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.159595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.160570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.160984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.263203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.264829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.271730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77c18-ed82-4a8f-b189-d29cd744335d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.317323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.317847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.318179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.318447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.319184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=fa271ae95240497e9be3175e44a2b545) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:54.319491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.320976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1cbcc-0db2-484f-acfa-6d9be4693939 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.326924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 345/664] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/role_assignments?user.id=fa271ae95240497e9be3175e44a2b545&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:54.337449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.337786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.338004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.365996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.382100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:54.383393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:54.397357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.435953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 320/665] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:54.437058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.523190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 346/666] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:54.528255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.586693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea71b28-4dcc-4a79-8e61-3ff71d7f93cb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.631940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.632615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.632969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.633307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.634258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:54.634624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.636317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.651063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1e943-4c05-4c5b-9381-6580009f7c12 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.656382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 321/667] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:54.662766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.687872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.689274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.689749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.690190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.694138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.702366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.704162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:54.719772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:54.720768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:54.721632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:54.721632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:54.721632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:54.724707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 347/668] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:54.728261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.731483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.791635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca044-fc64-4f2b-82e7-aa02c718718b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.829334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.830111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.830461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.830805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.850184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502431-6ee2-4c9a-b88d-a57169817f3f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:54.887699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:54.888329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:54.888610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:54.888876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:54.889647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:54.889950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.891311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.909253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 348/669] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:54.910125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:54.925788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:54.928171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:54.974541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:54.977313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.047413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6ae89-dc72-4fbc-8515-5fecb1c5f135 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.049754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 322/670] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:58:55.051318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.089535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.090168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.090443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.090714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.102238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:55.107116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:55.108118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:55.108946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 349/671] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:58:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:55.110852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.189926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de4d6-5ac2-4cbf-8fa8-27a10d1dff50 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.229260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.229885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.230159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.230428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.237587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.239093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.242012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.250415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76e8c-d863-40cf-89eb-2ab7740b93a3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:55.251425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:55.256506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 323/672] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:55.258187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.267022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.296640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.297477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.297751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.298021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.309789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.311232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:55.327394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:55.328421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:55.329896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:55.329896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:55.329896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:55.329896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 350/673] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:55.329896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.388296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c3a1-b6df-4766-af02-6ffcf849a26b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.425082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.426323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.427302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.427302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.428007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:55.428007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.429508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.443377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 324/674] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:55.444311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.461322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2ca3b-cbb9-4ab8-bcff-73b7b903cf3b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.510125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.513232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.514110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.514795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.515722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.516824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:55.517433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.520026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.538020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 351/675] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:55.539564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.551611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc894c7-8685-454a-99a9-318bd2198054 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.568934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.586341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.587490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.587490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.587490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.588135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:55.588433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.589897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.604212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 325/676] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:55.605164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.665915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174b00-f775-43e2-bcf5-cc2cd411fb33 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.706000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.706724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.707536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.707748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.710004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6357-061f-4730-a5d4-dac84f753c5e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.718553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.720188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.734788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:55.735214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:55.736204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:55.737093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:55.737093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:55.737093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:55.737093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 352/677] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:55.737792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.751999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.752641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.752918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.753185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.754789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bb59243400b44b8e998daaa4bd8f1ef0) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:55.754789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.756184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361375-78e9-4167-8d97-3cd32f7e1831 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 326/678] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/role_assignments?user.id=bb59243400b44b8e998daaa4bd8f1ef0&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afe206-28f9-4ca4-bbc7-2ac60c85f14e None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afe206-28f9-4ca4-bbc7-2ac60c85f14e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afe206-28f9-4ca4-bbc7-2ac60c85f14e None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 327/679] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:55.767843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afe206-28f9-4ca4-bbc7-2ac60c85f14e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.849778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:55.889985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04219-524d-491f-981e-91dc0596e0ee None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.907935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afe206-28f9-4ca4-bbc7-2ac60c85f14e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:55.944985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.945548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.945825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.946093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.953540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.954265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:55.954865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.954979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.955148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.955397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.956057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:55.956353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:55.959741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e60fc2cf-d153-4f47-b8a1-b2f3fce1e73e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:55.968644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:55.969529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:55.970432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 353/680] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:55.971375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:55.971481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:55.971887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:55.977997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 328/681] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:55.979455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:55.995427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:55.996629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:56.054112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.088340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 354/682] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:56.089214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.090299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d685a1c-0e92-4d72-85bf-41bae1236e98 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.165359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.186108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.186886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.187256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.187620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.188437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:56.188889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:56.190491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:56.219253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9d5a3-1170-4569-9ace-87d707b11295 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.220483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 329/683] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:56.236872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.267900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.270901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.271188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.271462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.287654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:56.289143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:56.308000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:56.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:56.309898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 355/684] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:56.319133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.433948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634e765-9124-4d1d-a97c-edca4606f62d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.446682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.478934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388171-2d73-4c00-87c9-61d3c72ed597 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.479398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.480051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.480329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.480602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.481335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=94f1f1b96cd6477581e1040d8d8e0529) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:56.481642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:56.483108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:56.489946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 330/685] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/role_assignments?user.id=94f1f1b96cd6477581e1040d8d8e0529&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:56.490805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.531453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.557349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.558085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.558362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.558632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.559375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:56.559678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:56.561185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:56.605233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 356/686] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:56.611422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.714971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a10ba-a885-423f-845d-83a9a0e74473 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.755441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.756028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.756310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.756580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.771519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-6f0bd20e-2598-47b0-9338-f455c69416fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:56.776524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:56.777446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:56.778269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 331/687] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:56.779687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef630a47-b8d1-402a-8378-ea3816d1a057 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.779687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef630a47-b8d1-402a-8378-ea3816d1a057 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.779687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef630a47-b8d1-402a-8378-ea3816d1a057 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.781093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 332/688] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:56.786949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef630a47-b8d1-402a-8378-ea3816d1a057 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:56.796459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420e87-13cf-46b7-a1c6-7de6672d5711 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:56.829101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.840852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:56.841472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:56.845553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:56.845553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:56.903808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:56.936321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:56.939137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:56.948899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef630a47-b8d1-402a-8378-ea3816d1a057 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.005787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.007143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.007941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.008424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.009486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:57.010290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.011932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea5ea2-e3ea-4e63-931a-d6a1a6d6b241 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.029270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 333/689] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:57.030757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.031285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.031645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.054001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:57.055541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:57.085709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 357/690] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:57.086893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.160378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 334/691] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:57.160378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.198125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.212649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.231650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d6f26-f614-4f24-804d-8bb446e61cbd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.306134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9ff30-0cbe-47c4-a3db-ea7546dd7c77 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.332660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.333365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.333642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.333909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.352748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.357759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.362341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.363725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.364020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.364284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:57.383199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:57.384322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:57.385751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:57.385751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.385751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:57.385969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.387309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.387546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 358/692] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:57.390856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:57.414692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:57.415701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:57.416606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:57.416606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.416606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:57.417504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 335/693] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:57.423933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.560978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.579306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335a40e-0929-4f03-95c0-8fc4e52a8e23 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.589417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10236b0-3236-4673-82d9-47bb87fa7819 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.612028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.618094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.619112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.619397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.619749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.620487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:57.626762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.626762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.631968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.632892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.633374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.633842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.634695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:57.635198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.636807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.641466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 359/694] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:57.642419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.653027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 336/695] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:57.654394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.772185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998b10-6181-498e-91ac-df5ae513cf32 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.782232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740b7d-0b15-44f3-9c7f-8dc984ddf7c4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:57.811284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.812393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.812899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.813419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.816599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.824356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.825985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.828310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:57.830018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:57.830520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:57.831118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:57.839645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:57.843060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 337/696] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:57.844028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.847405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:57.849415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:57.870026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:57.871246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:57.872281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:57.872281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:57.872281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:57.873296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 360/697] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:57.879143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:57.928139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:57.983990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fef8c-04ad-4bb6-8f70-8cb08105e974 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.001891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d533-d308-4315-b687-1126416532c2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.024050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.024685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.024960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.025223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.033379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.035382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.048518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:58.048771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:58.049610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:58.051164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:58.051644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.051644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.052385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.053838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:58.054384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.055506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 338/698] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:58.056486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.057485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879c9ff-9cf7-43c9-b1d6-41421aefe461 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.079718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 361/699] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:58:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:58.081614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277247-c7f2-464d-a28b-912c1ed41548 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.082679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277247-c7f2-464d-a28b-912c1ed41548 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.083172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277247-c7f2-464d-a28b-912c1ed41548 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.084771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 362/700] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:58.085749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7277247-c7f2-464d-a28b-912c1ed41548 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.127772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.176399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59919ba-c99f-4571-8feb-922887ef77e3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.203718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7277247-c7f2-464d-a28b-912c1ed41548 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.205454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.212946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.218372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.230268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 339/701] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:58.231149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.249193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.249833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.250047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.250331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.251023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:58.251319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.252791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa399ba-7c65-4ecc-849a-fecc22710f60 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.266054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 363/702] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:58.267096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.267379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.267646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.288692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:58.289941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:58.369274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99258-58b0-4563-9dda-f07876b4e117 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.372043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 364/703] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:58.372882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.395496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.423987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.425753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.426287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.426782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.485066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.506226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55c6bd-db75-44bb-a666-71688affc7d4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.520302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.522719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.552499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.553124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.553404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.553677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.554328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.555786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.652832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 340/704] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:58:58.654021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.665096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 365/705] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:58:58 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:58.666102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.675231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.785290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcca3a-b9b8-446c-8bd6-8166ff010199 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.818041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9116-e090-4b56-b493-19cf5b41dc48 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:58.827566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.830541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.830541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.830541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.830541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=d045b7cdd6024b6b82f3c4864c01e7d3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:58.830541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.832459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.847904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 341/706] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/role_assignments?user.id=d045b7cdd6024b6b82f3c4864c01e7d3&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:58.847904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.890880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:58.891684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.892049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.892390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.894823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:58.896705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9d8383a5-0208-4c5b-834a-520ee58cc3b0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:58.914058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:58.915224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:58.916929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 366/707] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:58.917715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc054f98-0c64-489e-9425-443c76637e90 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:58.917715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc054f98-0c64-489e-9425-443c76637e90 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:58.917715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc054f98-0c64-489e-9425-443c76637e90 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:58.918750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 367/708] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:58.921227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc054f98-0c64-489e-9425-443c76637e90 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:58.940313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.971270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:58.999255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18e4d-5500-4727-aefa-565f272575a9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.048970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.050520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.050874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.051216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.064677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.066308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-d3d8ecaf-5e7e-4597-96a6-815ebcdf85c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:59.083183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:59.084215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 342/709] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.086404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.086793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc054f98-0c64-489e-9425-443c76637e90 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.086875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.107316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:59.108600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:59.132750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.134137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.134714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.135242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.136206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:59.136667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.138807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.145837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 368/710] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:58:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:59.147065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.196868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 343/711] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:58:59.197754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.283664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:59.309668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:59.327059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30580-a04e-40f3-956b-5a05f4daa65c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.400931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687aaa-5a79-4ab3-869a-2e5c41aa5e0d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.404869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.405486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.405764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.406032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.406683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:59.406986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.408386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.431129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 369/712] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:59.438354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.494300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.494827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.495106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.495377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.496040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:59.496342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.497728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.513612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 344/713] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:59.529430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.637894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bec2-713b-4761-a43a-a99a995b6455 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.710333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.710987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.711263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.711535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.719616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.720670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd507-831b-4446-93cb-919f2561546b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:59.721227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.721941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:59.735465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:59.736501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:59.737385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:58:59.738879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 370/714] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:59.745152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.776260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:59.789383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:59.791388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:59.791388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:59.791612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] PATH_INFO: `/v3/services/network` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:59.792378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:59.793840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:59.813345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:59.814383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:58:59.815213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:58:59.822240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 345/715] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:59.826845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:59.937383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52917f-a218-482f-b41e-78af3c4ae072 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:00.001409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:00.001947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:00.002223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:00.002492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:00.003713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:00.004052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:00.005490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:00.019918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 371/716] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:00.029488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b43d8-08a9-419a-8a06-1a485bf7e628 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:00.034372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:00.066347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:00.067007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:00.067287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:00.067559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:00.068244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] RBAC: Adding query filter params (name=network) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:00.068548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:00.069953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:00.075406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 346/717] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:58:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:00.084970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:00.116602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:00.163977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18532c-c538-4e6c-97f7-dc1d548ac24b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:00.198390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:00.228955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:00.229575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:00.229851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:00.230120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:00.230849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:00.231154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:00.232596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938ced-772a-4adf-b222-c66d5245abc3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:00.245749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 372/718] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:00 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:00.267616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80612662-246e-4149-bb80-6a118a04cd7e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:00.307108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:00.307733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:00.308042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:00.308311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:00.308952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] RBAC: Adding query filter params (type=network) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:00.309251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:00.310704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f5dd-61c8-4bf7-bd5b-c849b27dbbdc None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:00.317123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 347/719] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:01.072104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ede6d-5e6e-45bd-92d2-ab90a5f4c7ce None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.072606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ede6d-5e6e-45bd-92d2-ab90a5f4c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.072681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ede6d-5e6e-45bd-92d2-ab90a5f4c7ce None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.074273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 373/720] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:01.079331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24396cb-bd69-4e09-8b5b-39ec673594c9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.079750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24396cb-bd69-4e09-8b5b-39ec673594c9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.079949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24396cb-bd69-4e09-8b5b-39ec673594c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.104821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a24396cb-bd69-4e09-8b5b-39ec673594c9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:01.106017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a24396cb-bd69-4e09-8b5b-39ec673594c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:01.196930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 348/721] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:59:01.512936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee3e-5921-4950-8a3d-540ed14b12bd None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.513357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee3e-5921-4950-8a3d-540ed14b12bd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.513469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee3e-5921-4950-8a3d-540ed14b12bd None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.515261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 374/722] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:01.522480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9723-c3de-44d3-958b-1a94186e6259 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.522765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9723-c3de-44d3-958b-1a94186e6259 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.523040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9723-c3de-44d3-958b-1a94186e6259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.545956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-30dc9723-c3de-44d3-958b-1a94186e6259 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:01.547150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-30dc9723-c3de-44d3-958b-1a94186e6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:01.602415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:01.636485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 349/723] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:59:01.649326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ee3e-5921-4950-8a3d-540ed14b12bd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:01.776920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ee3e-5921-4950-8a3d-540ed14b12bd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:01.806457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:01.827776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:01.828484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.828763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.829033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.829147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4e0a-dc3b-43d8-9dc6-7d40ab6d6c6c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.829433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4e0a-dc3b-43d8-9dc6-7d40ab6d6c6c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.829709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4e0a-dc3b-43d8-9dc6-7d40ab6d6c6c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.832120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 350/724] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:01.837333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:01.837556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:01.837805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:01.845479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:01.851175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:01.852195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:01.853150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 375/725] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:59:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:01.860537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:01.892922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:01.894737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:02.009907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6846d28-2100-48cc-a77c-15593a416476 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.015991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 351/726] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:59:02.055826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.070714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.071336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.071637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.071954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.072632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:02.073165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.074600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.095997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 376/727] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:59:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:02.102898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.185517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:02.241093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c760cb-b92a-4d28-8c63-812c97820c75 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.260223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:02.299252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac56b00-5f28-48cb-bf71-5600c803d630 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.305098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.305742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.306070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.306291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.323038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:02.328201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:02.330241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:02.332847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 352/728] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:02.337196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.365863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.366774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.367052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.367330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.383676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.385347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:02.405098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:02.407209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:02.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:02.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:02.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:02.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 377/729] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:02.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.464302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7f2a-f82f-44d8-81a8-047e37539674 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.516733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.517343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.517617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.517888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.518545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:02.518955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.520392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.528967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 353/730] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:02.529778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.552245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:02.559230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f208a-db4c-4862-acdc-f7388764a574 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.563800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:02.593956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.594566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.594874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.595116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.595819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.597911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.643049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2d9c-59f7-4ba5-889a-f01fd7ffba83 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.676824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.677424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.677700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.677970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.678602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:02.678902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.680316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2169bf-6821-4b0f-987f-e1bbb3d6f27b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.691456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 354/731] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:02.692682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1755d-cce2-43e5-a652-5b4c59af6292 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.692961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1755d-cce2-43e5-a652-5b4c59af6292 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.693233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1755d-cce2-43e5-a652-5b4c59af6292 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.696291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 355/732] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:02.699471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1755d-cce2-43e5-a652-5b4c59af6292 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.703028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 378/733] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:02 2022] POST /identity/v3/services => generated 251 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:02.705371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.807028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:02.828313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1755d-cce2-43e5-a652-5b4c59af6292 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.852599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526ba6-978e-4995-a406-6da150ae0457 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:02.869266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.869891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.870161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.870423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.881606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.883353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.896143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:02.896782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.897061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.897333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-8dd883cb-8c03-43e0-a234-0fe7f362a465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:02.899908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:02.901028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:02.902124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:02.902124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:02.902124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:02.902124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 356/734] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:02.902400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e25d-a119-4f6e-8225-e95e7a8eea8d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.902490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e25d-a119-4f6e-8225-e95e7a8eea8d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.907171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e25d-a119-4f6e-8225-e95e7a8eea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.907331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:02.907331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-aac5990a-0cfb-45ba-b3d7-045e81f89196 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:02.919266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:02.920267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:02.921200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 379/735] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:02.922873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c351-4b40-4815-9472-66fb60452797 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:02.922873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c351-4b40-4815-9472-66fb60452797 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:02.922873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c351-4b40-4815-9472-66fb60452797 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:02.923501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 380/736] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:02.927888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52c351-4b40-4815-9472-66fb60452797 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:02.929544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2448e25d-a119-4f6e-8225-e95e7a8eea8d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:02.930723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2448e25d-a119-4f6e-8225-e95e7a8eea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:02.945337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.003740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 357/737] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:59:03.005055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.005325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.005616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.025765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:03.027242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:03.056123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52c351-4b40-4815-9472-66fb60452797 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.095342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.096641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.097223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.097736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.098532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:03.099283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.100812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.112226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.113882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 358/738] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:59:03.114370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 381/739] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:03.119597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.120173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.202720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.230449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e191b2-df42-402a-9fca-f8619613fbfc None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.250687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a06b5-acba-4155-87a1-9f9980a2a675 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.268094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.269776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.270315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.270789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.278450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.280216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:03.293171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:03.294128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:03.295099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:03.297603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 382/740] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:03.303632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.328713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.329344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.329625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.329908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.330664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:03.330969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.333141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.365541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 359/741] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:03.366679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.417505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b1d2-605b-4662-a09e-ca1a289f3c1e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.441033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.455105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.456031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.457115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.457536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.458424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:03.458890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.460516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.462515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.485414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 383/742] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:03.486437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.498183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce404523-8c23-48ca-9737-a33601b5a28c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.538481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.539193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.539471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.539739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.551494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:03.556620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:03.557534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:03.558371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 360/743] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:03.559056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.600559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040829-6dd9-4d62-a97a-a1866841281d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.639448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.639833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.640196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.640470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.677967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe0a6-ebd8-4e94-8467-d486aa7ffd81 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.692716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.704139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.714185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.714529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.714798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.715077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.715351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.716014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.787317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.788822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:03.827944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 384/744] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:59:03.827944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.899645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 361/745] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:59:03.900866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:03.917087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.926267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:03.934733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9034-08d7-4838-8fbf-82df26afe2af None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:03.969026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:03.969684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:03.969957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:03.970247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:03.970894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:03.972346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.007552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeaa92-a8ef-4661-b2eb-85ff173d30df None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.041275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.041890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.042163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.042437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.043091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:04.043393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.044867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.052153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 385/746] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:03 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:04.053692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.054195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 362/747] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:04.055066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.153403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.181600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344863c-44f0-477a-ba86-68b895fab4ec None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.224125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab261ee-c0b8-4ba1-a602-a7cba9ad007d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.228540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.230100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.230100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.230100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.264905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.265574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.265851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.266121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.278482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.280031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.292332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:04.298362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:04.299385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:04.304672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:04.304672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.304672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:04.304672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 386/748] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:04.304672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.362980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.364478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:04.380160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:04.381366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:04.382483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:04.382483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.382483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:04.382483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 363/749] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:04.382483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.422842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69492f-0354-435c-925a-b1874b0d3acb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.460717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.461456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.461827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.462159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.477246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.478953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:04.498470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:04.499688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:04.500825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:04.500825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:04.500825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:04.502891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 387/750] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:04.503964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.513349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90c07-4e5a-42ae-b67a-50dda908865a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.547671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.558069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.567822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.568387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.585447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 364/751] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:04.585447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.622664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aec93-1a88-45e7-b325-a08be10bf068 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.662197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.663436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.664301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.665229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.666470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:04.667245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.669592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.691189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 388/752] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:04.691189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.722537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d4f5e-c688-4f25-b4ac-19b90ab2bfea None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.759326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.759958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.760240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.760512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.761187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:04.761487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.762863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.775226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 365/753] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:04.776074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.802719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead70d-69a3-4199-83cc-1641135cd775 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.818349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.829665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:04.845759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.846397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.846678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.846981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.854157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.855603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e7576e28-74c2-4322-9fc6-b825d3509a2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:04.868117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:04.869000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:04.869902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 389/754] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:04.871085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ebb1d-0caf-4101-975b-74ce605a139f None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.871085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ebb1d-0caf-4101-975b-74ce605a139f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.871085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ebb1d-0caf-4101-975b-74ce605a139f None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.876419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 390/755] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:04.876419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ebb1d-0caf-4101-975b-74ce605a139f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.890636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116975b-9029-4e69-9b6c-1e8a3dff7247 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:04.929642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:04.930272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:04.930829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:04.930829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:04.937759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:04.939155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:04.958866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:04.959773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 366/756] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:04.960796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:04.986415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ebb1d-0caf-4101-975b-74ce605a139f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.023913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.024779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.025233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.025498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.033014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.034434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c5f1717d-29e0-47ab-afaf-7090950e535b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:05.045472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:05.046413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:05.047438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 391/757] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:05.048061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.048893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.048893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.066724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:05.067912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:05.075138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15144b-f8a1-4a40-93d6-0ea8a6531f78 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.085542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.094579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.114073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.117782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.117782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.117782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.117782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.117782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.119168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.132346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 367/758] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.133348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.141273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 392/759] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:05.141974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.259304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3e55-777d-4361-8a07-8c4700e734e2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.264642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421d55-ad3d-4b35-b427-76332c444247 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.301347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.302004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.302284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.302551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.303269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.303560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.305029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.305833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.306862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.307275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.308544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.309390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.309839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.311525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.325470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 393/760] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.326262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.327723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 368/761] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.331962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.349361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.370439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.429638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3367-091c-4796-ad14-c94f6c65d727 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.451213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe21e-eb14-4ceb-a925-adbd09382810 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.465647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.466251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.466526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.466830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.467537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=b86144ce73784001bc405a8e2a4b1e0d) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.467833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.469275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.474912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 394/762] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/role_assignments?user.id=b86144ce73784001bc405a8e2a4b1e0d&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.475685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.489344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.490376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.490863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.491356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.504485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.506444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:05.524083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:05.525049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:05.525914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:05.525914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:05.525914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:05.528216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 369/763] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:05.529461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.599160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.619408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e692b-ed02-4fcf-83be-49d154c34673 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.621921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.644083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2be64-f095-4e3e-aca2-db2598241f93 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.663547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.665740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.665740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.665740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.665740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=d045b7cdd6024b6b82f3c4864c01e7d3) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.665740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.667508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.673738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 395/764] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/role_assignments?user.id=d045b7cdd6024b6b82f3c4864c01e7d3&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.674671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.683823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.684522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.684800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.685070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.685660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.687047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:05.697575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:05.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:05.699438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:05.703891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 370/765] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:05.705717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.815655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660ca6-0ef9-48b8-984d-00a8e01f4e4f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.818397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0b44-5b88-4478-998f-310e041c6c55 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:05.855667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.856761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.857335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.857789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.858697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.859194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.860115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:05.860815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:05.861154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:05.861488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:05.861746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.862387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:05.862802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:05.864489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:05.874930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 371/766] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.876300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 396/767] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:05.876626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.889026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:05.889615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:05.909026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:06.025290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ebe4-9383-40a9-b7a2-45dd38c63acd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.045793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727d7e-12b9-42eb-ace3-73a3bc860768 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.094674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.095200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.095478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.095749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.096409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:06.096739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.098242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.105562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.106200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.106481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.106752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.112924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 372/768] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:06.117186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.119305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:06.122396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:06.145436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:06.146395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:06.147308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 397/769] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:06.155836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:06.219471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:06.249306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d721b-3305-4160-9117-21aa60d2dc38 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.285004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.285612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.285887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.286158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.286848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] RBAC: Adding query filter params (interface=public, service_id=b7b78eb13a6d44149dcc1066b050d3cd, region_id=RegionOne) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:06.287151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.288407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:06.288848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.306147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 373/770] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:06 2022] GET /identity/v3/endpoints?service_id=b7b78eb13a6d44149dcc1066b050d3cd&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:06.311651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:06.344371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a78-c3f0-4ebb-b1d7-55ce990bbd34 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.382488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.383439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.383966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.384544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.385550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:06.386064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.387677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a18a2-fa22-46b9-abd1-8515fbbece79 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.401769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 398/771] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:06.403422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9c2a9-5ba7-4848-a94b-23ca2f8fb0e2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.403954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9c2a9-5ba7-4848-a94b-23ca2f8fb0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.404617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9c2a9-5ba7-4848-a94b-23ca2f8fb0e2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.411128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 399/772] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:06.412328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9c2a9-5ba7-4848-a94b-23ca2f8fb0e2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:06.421111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74defce-7d2c-4ca0-b1dd-0cb504e321d9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.456864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84568518-8255-4a01-b244-05f544c93848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.457469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84568518-8255-4a01-b244-05f544c93848 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.457741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84568518-8255-4a01-b244-05f544c93848 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.458009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84568518-8255-4a01-b244-05f544c93848 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.458854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84568518-8255-4a01-b244-05f544c93848 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.460098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84568518-8255-4a01-b244-05f544c93848 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.462015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:06.467937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 374/773] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:06 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:06.467937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.467937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.467937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.489031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:06.490487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:06.565477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 375/774] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:06.584701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9c2a9-5ba7-4848-a94b-23ca2f8fb0e2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:06.626519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:06.642307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:06.642307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:06.642307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:06.642307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:06.649538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=bf0af6b468c7442e818efec1543b7eef, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:06.651028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:06.719112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 400/775] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:06 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:59:06.913543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:07.039317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938ede0-15fd-4cd6-84d3-b2802c33b9f3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:07.070230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:07.076583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:07.077360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.077642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.077915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] PATH_INFO: `/v3/services/network` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.078664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:07.088545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:07.089527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:07.090458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:07.091268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 376/776] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:06 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:07.097645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:07.253289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d22a-0b7d-4fab-a200-0d8c18648157 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:07.305953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:07.307345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.307819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.308329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.309151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:07.309662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:07.311216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:07.325854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 401/777] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:07.332149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:07.485196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb78f82-d2ca-44f8-baf8-f3fddc599d84 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:07.527080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:07.527653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.527961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.528230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.528865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:07.529168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:07.530545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bcb16-d37e-472c-a2f2-16d64c7243a7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:07.535913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 377/778] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:07.541703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:07.624458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:07.644683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831b73-ddd6-4343-ad62-2f4f4db2d6ff None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.644985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831b73-ddd6-4343-ad62-2f4f4db2d6ff None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.645238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831b73-ddd6-4343-ad62-2f4f4db2d6ff None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.646837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 378/779] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:07.655535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5c3f9-981f-4580-bdca-7314719f3045 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.655811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5c3f9-981f-4580-bdca-7314719f3045 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.656109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5c3f9-981f-4580-bdca-7314719f3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.678862 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0577c-e5d6-4efe-a9cb-79a7429d37c1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:07.696074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-44d5c3f9-981f-4580-bdca-7314719f3045 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:07.700258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-44d5c3f9-981f-4580-bdca-7314719f3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:07.744983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:07.746750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.747036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.747314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.748046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] RBAC: Adding query filter params (interface=public, service_id=1cd9c5de63bc4ea9abf3d468be5d2311, region_id=RegionOne) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:07.748361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:07.753840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:07.769300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 402/780] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:07 2022] GET /identity/v3/endpoints?service_id=1cd9c5de63bc4ea9abf3d468be5d2311&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:07.779577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:07.812283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:07.842411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 379/781] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:07.921629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd551-43cd-40a6-9dd2-cd8731676616 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:07.939078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:07.958069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:07.959251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:07.959251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:07.959251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:07.959846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:07.961249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931f1a4-663b-49ea-9d80-3405672c06eb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:07.966825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 403/782] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:07 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:08.038243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec218-4948-4446-be5a-f51a7ccef460 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:08.038441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec218-4948-4446-be5a-f51a7ccef460 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:08.038698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec218-4948-4446-be5a-f51a7ccef460 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:08.040085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 380/783] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:08.046021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:08.046330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:08.046581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:08.076139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:08.077380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:08.177071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 404/784] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:08.202060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ec218-4948-4446-be5a-f51a7ccef460 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:08.339786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ec218-4948-4446-be5a-f51a7ccef460 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:08.392759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:08.393408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:08.393866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:08.394114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:08.416238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:08.421503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:08.431984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:08.432817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 381/785] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:59:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:08.433510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:08.499512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:08.578485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:08.639315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c953e-0b6a-44d5-94c9-3697a6ac8192 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:08.642132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c9d6-2425-4c2a-b4e1-5625e9ada249 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:08.668214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:08.707364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:08.708044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:08.708324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:08.708597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:08.709700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:08.710313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:08.710585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:08.710852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:08.711531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:08.711831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:08.713243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:08.732643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 405/786] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:59:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:08.733302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:08.738493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:08.739449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:08.740359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:08.741004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:08.744796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 382/787] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:08.749766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:08.920259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b5bb-7684-47e8-ae9b-03d6e100a1ec None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:08.961033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0d707-790b-49e2-ab0b-ffcb85d9e825 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:08.978286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.004629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.005291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.005587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.005838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.017361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.019386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.028200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.028886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.029166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.029436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.030211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:09.030515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.031910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:09.035155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:09.036667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:09.038144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:09.038144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:09.038144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:09.038144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 406/788] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:09.043919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.057251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 383/789] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:09.069313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.101810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.197681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935b3ed-4c84-4e77-ae39-e1660023e9b8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.215695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede73c-82a7-4605-b56e-08a6bebf7ebc None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.240952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.241938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.242365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.242783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.256937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.258759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-2535a149-0b84-405b-8f49-43620aa2c671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:09.276168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:09.277262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:09.278153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:09.278153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:09.278153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:09.278738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 384/790] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:09.279786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e3b1-f552-4164-a270-4f6c8fcf1ccb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.280120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e3b1-f552-4164-a270-4f6c8fcf1ccb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.280392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e3b1-f552-4164-a270-4f6c8fcf1ccb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.285742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 385/791] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:09.285742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e823a0-4925-4258-982e-ed14f0fab6bb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.285742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e823a0-4925-4258-982e-ed14f0fab6bb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.285742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e823a0-4925-4258-982e-ed14f0fab6bb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.285742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 386/792] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:09.286396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e823a0-4925-4258-982e-ed14f0fab6bb None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.327689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.396327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.397171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.397619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.398061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.399215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:09.399555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.401784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8e190-706f-4fb7-b5dd-702d9ee0dd21 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.426989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 407/793] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:09.426989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.426989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.426989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.444386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:09.445688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:09.471791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e823a0-4925-4258-982e-ed14f0fab6bb None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.507755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.508544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.508701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.509027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.509684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:09.509984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.511395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d6212-3139-4330-be0a-c3d8cc5008e5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.524467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 387/794] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:09.525843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.526123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.526391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.540021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 408/795] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:09.540900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.557642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.558683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:09.560694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:09.664243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd544eb-fe28-4763-bb9e-0c62a90975cf None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.675878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 388/796] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:09.675878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.698214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.736787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.737947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.738222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.738484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.753693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.755169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:09.774611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:09.775620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:09.776589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:09.777658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 409/797] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:09.783190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:09.886289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:09.897973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fdca6-0059-40c6-8c69-cdbc524d5df0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.915783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611889f1-98e1-487d-8e73-7f7411447f80 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:09.953517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.954311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.954591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.954863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.956794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:09.957486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:09.957835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:09.958173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:09.958975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:09.959397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.961013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.967531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:09.969527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:09.983827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:09.984851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:09.986405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 389/798] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:09.988387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.010091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 410/799] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.012080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.081881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.121199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea1856-3fc1-45e3-a1ba-57737588edfb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.141814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f45f5-88f4-4cb4-8bbc-fa9d671a10fb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.168610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.169248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.169532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.169835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.177685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.182276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:10.183178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.183846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.192049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.192049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.192049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:10.192049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.192049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:10.192414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:10.193337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 390/800] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:10.194157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.223578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 411/801] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.225690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.323020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3293e1-4337-41df-8de5-00a403201bcc None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.350091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff308c1a-45d2-4c03-b360-b3b58d954db1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.367410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.373014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.373652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.373925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.374194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.390655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.392811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.394380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.395019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.395293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.395585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.396590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.398908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:10.410163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:10.411199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:10.412051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 412/802] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:10.413484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:10.414807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:10.415950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:10.417120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:10.417120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.417120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:10.419494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 391/803] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:10.420683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.485866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.530553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a56d6-308d-4c4b-b744-5494987c556a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.534593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611fbe-caf2-4413-80e3-a6f70d90714f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.564548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.565151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.565998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.566267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.566923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:10.567218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.568489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.568603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.569117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.569391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.569660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.577224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 413/804] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.578002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.581623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.583114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:10.598343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:10.600627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:10.601535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:10.601535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.601535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:10.601535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 392/805] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:10.601535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.639398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.684004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107e5e-02b0-4d0a-829a-956a8f1e8f65 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.700393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.704187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f184002-4ba8-4dc4-996d-f3e284e7f92e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.718341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.718985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.719255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.719518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.720179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:10.720476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.721865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0d8d-fc3c-47e7-a0cf-5ae42632045b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.726635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 414/806] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.728123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c48a-1690-415d-adee-2c968040dbf0 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.728446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c48a-1690-415d-adee-2c968040dbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.728715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c48a-1690-415d-adee-2c968040dbf0 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.729970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 415/807] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:10.730660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96c48a-1690-415d-adee-2c968040dbf0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.749099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.749702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.749976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.750561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.750878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:10.751179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.752684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.764416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 393/808] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.765228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.844397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96c48a-1690-415d-adee-2c968040dbf0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.877747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:10.885802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcba196-3811-45d6-ab4a-ef5f893d8b00 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:10.891272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.891920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.892195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.892461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.903145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.904782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:10.919902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:10.921009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:10.921922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:10.921922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:10.921922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:10.921922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 416/809] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:10.922147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:10.922236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:10.922315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.922669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.922669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.923306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:10.923613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:10.925648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441631-cc8a-4b17-a540-4b9e536e7eda None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:10.937240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 394/810] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:10.938534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:10.938820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:10.939087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:10.958263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:10.959456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:10.965408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.047663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 395/811] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:11.048535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.056179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39876f42-70ae-4732-b540-f04f1a96e50d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.096481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.097725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.098359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.098359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.099463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:11.100025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.101310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.113031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 417/812] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Aug 29 18:59:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:11.113031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.158405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.176800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d1968-28e2-4076-801f-5ca54b21f330 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.222802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.224548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.225079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.225601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.234827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.236318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.236605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.241839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060152-3719-4c20-bdd0-21a4150ad271 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:11.249471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:11.251099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 396/813] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:11.252151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.288760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.289634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.290056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.290502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.291306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:11.291750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.293314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.306450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 418/814] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:11.307691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.374436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300218c3-8ab4-4412-a6f2-8ceab9db56af None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.411110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.411746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.412152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.412431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.417851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc03cd-017a-4be8-9b1a-596cf085622d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.419202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.420699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:11.432567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:11.433572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 397/815] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:11.434490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.445809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.456728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.457389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.457646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.457897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.458570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.467752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.545697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be116d2-6639-4bbf-afcc-c141b8c07d44 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.563073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 419/816] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:11.576941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.595734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.596407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.596707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.596981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.597753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:11.598062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.599837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.606429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.622586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 398/817] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:11.624297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.712846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.736170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddbba6-dbd8-4969-903e-8bb3db5ffe3a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.762785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadf254-b215-4865-a86d-13bceff6e559 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.781522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.782161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.782436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.782707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.791355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.792850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.803377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.804382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.804857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.805313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] PATH_INFO: `/v3/services/network` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:11.805417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:11.806276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:11.807188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 420/818] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:11.807842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.808785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.810230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:11.821421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:11.822501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:11.823339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 399/819] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:11.824636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.898157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:11.930261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed9c47-3b43-4f8e-a748-7590c0994d88 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.953220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2cfc8-13db-4ceb-948f-ca107078d958 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:11.969674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:11.970712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:11.970996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:11.971270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:11.972630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:11.972939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:11.974642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:11.993403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 421/820] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:11.993403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:11.999458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.010147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.010817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.016411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.016411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/b86144ce73784001bc405a8e2a4b1e0d/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.037130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=b86144ce73784001bc405a8e2a4b1e0d, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.038819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.111198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5583c5-42a2-4905-bfec-6393aa0d787f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.135886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 400/821] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:11 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/b86144ce73784001bc405a8e2a4b1e0d/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:59:12.135886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.156598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.162389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.163148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.163499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.164012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.164864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:12.165235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.166784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.192946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 422/822] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:59:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:12.194211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.269211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb6155-cc39-4204-b6e0-0d3c8ec7cb47 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.312142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.313019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.313437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.313901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.314670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] RBAC: Adding query filter params (name=network) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:12.315655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.317148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.322876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 401/823] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:12.323766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.354463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758817c-a329-4ed9-94c2-a5256e16bbbb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.382748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.395492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.397457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.397457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.397457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/d045b7cdd6024b6b82f3c4864c01e7d3/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.399785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=d045b7cdd6024b6b82f3c4864c01e7d3, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.401925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.450151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c4163-33cb-4c9a-8715-fd41066e233f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.486952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.487586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.487957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.488282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.504690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 423/824] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:12 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/d045b7cdd6024b6b82f3c4864c01e7d3/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:59:12.506239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.585125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.650236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.651730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.661989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1896ee-d6d9-41e0-bdd4-f1daa3807462 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.706727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.707989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.708890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.709640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.710666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] RBAC: Adding query filter params (type=network) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:12.711194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.712848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:12.723486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 424/825] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:12.728671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.738733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.860559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 402/826] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:12.869978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:12.892390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44ccc-71da-4047-b0d9-e7c344eade08 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:12.909869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:12.931951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:12.932716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:12.933092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:12.933466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:12.934281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:12.935833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.016598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 425/827] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:59:13.017460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47972e-2889-4fd8-aaf9-10cc5988b8c0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:13.071138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:13.071796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.072098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.072375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.076486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:13.085850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:13.087400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:13.105883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:13.106913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:13.107785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:13.107785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:13.107785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:13.107785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 403/828] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:13.113725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:13.280984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b864bc-d4ff-4089-b02f-f51e7f9b5d41 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:13.321536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:13.322164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.322433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.322701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.323355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:13.323655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:13.325059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.339317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 426/829] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:13.345861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:13.474996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714a3b-cbf3-4598-9108-c04cb8e5262b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:13.512453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:13.513091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.513363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.513629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.521238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:13.522699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-d0da62bb-eb33-471a-a072-dada9be5332c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:13.536881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:13.537822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:13.538724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 404/830] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:13.544058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:13.578756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:13.668301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cf7ee-4b83-4c06-9e8a-be3c4df816b5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:13.704864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:13.705473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.705929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.706704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.707418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:13.707705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:13.709171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.720720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57262a-049c-40be-bb09-6b1676cd1beb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.722329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57262a-049c-40be-bb09-6b1676cd1beb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.722329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57262a-049c-40be-bb09-6b1676cd1beb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.723717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 405/831] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:13.724493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 427/832] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:13.729298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.729568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.729778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.730486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:13.752855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:13.754556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:13.818856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:13.856241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:13.864065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 406/833] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:13.874382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38139e60-9447-4d20-8b07-a492e348de54 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:13.945103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:13.946046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:13.946466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:13.946892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:13.948114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:13.948585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:13.951087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:13.962003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 428/834] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:13 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:14.245754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:14.370446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad5cdc-b5cf-4b5f-a451-0ef60ba750e9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:14.407936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:14.408575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:14.409025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:14.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:14.409735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:14.411087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:14.420598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:14.429886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:14.436099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 407/835] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:14.438032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:14.570603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046877e-cc37-435f-9f06-a16a32632cb4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:14.635608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:14.636846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:14.637316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:14.637774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:14.638863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:14.639377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:14.641904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f2985-00a7-46b8-a02e-8585c0bc0a5f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:14.651958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 429/836] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:14.661639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:14.685089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678b18-997d-4209-8739-a8eaec27fe58 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:14.685744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678b18-997d-4209-8739-a8eaec27fe58 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:14.686590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678b18-997d-4209-8739-a8eaec27fe58 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:14.689483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 430/837] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:14.697776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b576ea-e14b-4d67-8d85-12efc74c2dd0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:14.698199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b576ea-e14b-4d67-8d85-12efc74c2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:14.698643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b576ea-e14b-4d67-8d85-12efc74c2dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:14.729144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-55b576ea-e14b-4d67-8d85-12efc74c2dd0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:14.731212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-55b576ea-e14b-4d67-8d85-12efc74c2dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:14.732424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:14.757980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:14.799201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c63f-d455-442f-ac25-bd531084a9a0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:14.837919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:14.838461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:14.838740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:14.839016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:14.839654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:14.839985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:14.841447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf36af9-950b-4288-bf88-b6dc6c655db5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:14.856050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 408/838] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:14.867986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 431/839] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:15.125661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e84d-9547-471f-9ff4-4de09b86eb3e None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.125661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e84d-9547-471f-9ff4-4de09b86eb3e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.125661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e84d-9547-471f-9ff4-4de09b86eb3e None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.125661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 409/840] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:15.137148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.137148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.137148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.158565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:15.161819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:15.226850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e84d-9547-471f-9ff4-4de09b86eb3e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.291896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 432/841] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:15.395457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e84d-9547-471f-9ff4-4de09b86eb3e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:15.409537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:15.459279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:15.460457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.460961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.461455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.481074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:15.491408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:15.492407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:59:15.493231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:15.494486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 410/842] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:15.500636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.587215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:15.611779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc0cb6-b0c8-4dd2-9978-17764ae10dc1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:15.651141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:15.652190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.652626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.653036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.653661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.654028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:15.654468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:15.656016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:15.667804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 433/843] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:15.674678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.787739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26a864-e802-4568-82dc-463661405ea5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:15.795786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8b34-9faa-4ee9-adfe-9e3c3ca78ce1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:15.832543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:15.833192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.833476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.833751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.837179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:15.838111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:15.838586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:15.838997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:15.847752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:15.850289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:15.852687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:15.854718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:15.856248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:15.857370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:15.858227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:15.860567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 411/844] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:15.867978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:15.872161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:15.873102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:15.873990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:15.873990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:15.873990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:15.873990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 434/845] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:15.882610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:15.905700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.001875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ddc-f772-436a-8b9f-da4ef2688d28 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.005009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c689ef-864f-41eb-b7cd-9084cdde9afd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.039701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.040367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.040647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.040925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.041587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:16.041981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.043428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.046335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.047288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.047771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.048324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.049295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:16.050538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.052261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.054274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 412/846] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:16.059371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.072213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 435/847] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:16.075222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.136261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.169798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.185732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678a5c-557e-4df9-b792-d7e05a3abe60 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.196703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c97e40-7688-4b14-ae6a-c8a7bd73f077 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.246333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.247410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.247932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.248419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.251473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.253210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.253797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.254698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.265066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.266866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.267383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.268705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:16.283364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:16.285003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:16.285926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-6ccbd3be-507f-44e5-9725-4aff65a4f1ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:16.287537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:16.288474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:16.289340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:16.289340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:16.289340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:16.289340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 413/848] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:16.290471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aa2f-96dd-4fa6-8eda-6750e2e3087f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.295341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aa2f-96dd-4fa6-8eda-6750e2e3087f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.295341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aa2f-96dd-4fa6-8eda-6750e2e3087f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.295341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 414/849] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:16.296109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 436/850] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:16.296876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1aa2f-96dd-4fa6-8eda-6750e2e3087f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.297558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.468516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e4eaa-5d4f-4d2a-8567-1262b2501f1a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.469453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1aa2f-96dd-4fa6-8eda-6750e2e3087f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.484863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.512467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.513684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.514111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.514294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.514554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.514697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.514838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.514975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.515518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:16.515801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:16.516009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.516271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.517266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.518566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcfbea-81e8-466b-8f67-2ff87243f489 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.532442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.543775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 437/851] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:16.543775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.543775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.543775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.551311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 415/852] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:16.552637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.566919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:16.568584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:16.664800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 438/853] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:16.672959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.711966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b127e-09a6-4d7f-8d72-24a54cabe75e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.778941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.779993 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.780699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.782678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.800233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.802945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.819213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186c2a-beb8-4886-ba16-0c926e16597f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:16.821910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:16.823315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:16.824487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:59:16.825566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:16.828697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 416/854] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:16.837271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:16.869016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:16.869788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:16.870266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:16.870772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:16.958489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:16.961223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:16.972340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.023874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff25dd-36b5-42a1-bc7c-552b2d6bb76e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.077787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.078639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.079115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.079588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.081599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:17.082161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.084445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.110965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 417/855] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:17.111555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.146063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 439/856] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:17.147136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.149796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.284316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319e01-c00a-4172-b5ae-387d3d737696 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.304195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.319750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a047-2f9d-47b9-9422-9bc308d691db None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.322849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.323489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.323771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.324068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.324666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.327118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:17.336243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:17.337772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:17.338638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:17.340732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 418/857] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:17.342568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.376212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.376831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.377107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.377874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.456035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.457983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.458969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04526f-6561-412f-a3cd-eda35e0462e7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.460172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.496216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.496823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.497099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.497373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.510645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.512172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-44cc36c1-f725-41a4-af60-8cb2c94b093b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:17.528849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:17.529892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:17.530789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:17.530789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:17.530789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:17.530789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 419/858] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:17.531317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988b971-5e08-4972-8901-c28cebede6b9 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.531506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988b971-5e08-4972-8901-c28cebede6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.531760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988b971-5e08-4972-8901-c28cebede6b9 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.533281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 420/859] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:17.536750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988b971-5e08-4972-8901-c28cebede6b9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.571888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.577176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 440/860] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:17.578115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.654367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988b971-5e08-4972-8901-c28cebede6b9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.689299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540e0c1-f93f-4aec-8326-f5802652f894 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.691104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.697000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.697831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.698112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.698612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.699484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:17.699795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.701278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33512d4b-3cec-4c48-adc6-de2ed479a308 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.707136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 421/861] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:17.708493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.708981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.709292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.730196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:17.731422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:17.732047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.732958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.733479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.734086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.735155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:17.735968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.739297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.752657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 441/862] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:17.753521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.834147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 422/863] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:17.835123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:17.874077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:17.897486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5b852-e44a-4138-bf9a-33b8d365536b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:17.956589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:17.957754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:17.958102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:17.958588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:17.979779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:17.982260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:17.990329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39548cf8-3f23-4b50-b649-6cf28a0cc0e3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:18.010838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:18.012785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:18.014505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:18.014505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:18.014505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:18.016388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 442/864] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:18.018733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.033498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.034155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.034466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.034735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.035325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:18.035636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.037057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.042581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 423/865] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:18.043268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.060114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:18.203617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:18.219235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1724ca-7d50-48e0-a2f6-8911e80d8fea None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.220667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3214107-927e-44c5-808d-41fd3cd3100c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.269802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.270543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.270892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.271278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.278753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.279537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.279991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.280426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.281276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:18.281703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.283264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.283645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.286983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.300290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 424/866] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:18.301737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:18.309340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:18.310741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:18.311629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:18.316384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 443/867] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:18.317429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.430724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:18.472494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504514f1-a297-4b22-888a-9242641b2121 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.480374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b560014-a046-481f-ae69-dfd54dc4bc83 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.525834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.526716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.527054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.527381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.536567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.537570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.538027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.539110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.539988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.540482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.541765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.544252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:18.550443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:18.560314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 444/868] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:18.562064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.639985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 425/869] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:18 2022] POST /identity/v3/services => generated 251 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:18.639985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.675389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:18.726667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d611-b21e-46bb-a498-65ac3ed7b26d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.810065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:18.822550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.823992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.824579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.824735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.825824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:18.826281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.830661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.852044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187bfbe-b67b-4bc8-bf89-d4e6b27ed16a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:18.868054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 445/870] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:18.873162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:18.905949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:18.906726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:18.907037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:18.907278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:18.907994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:18.908300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:18.909841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:18.939032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 426/871] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:18.942122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:19.019282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd7a0-4c35-4266-a228-b973a2ed41b8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:19.059770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:19.060429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:19.060705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:19.060976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:19.061712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=b86144ce73784001bc405a8e2a4b1e0d) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:19.062018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:19.063648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba1162-8c44-43f4-8381-164b72d319ff None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:19.070577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 446/872] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/role_assignments?user.id=b86144ce73784001bc405a8e2a4b1e0d&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:19.164391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:19.175416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d284fd-4744-4117-ac5c-7afed8b6de72 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:19.232420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:19.233118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:19.233438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:19.233719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:19.234463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=d045b7cdd6024b6b82f3c4864c01e7d3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:19.234774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:19.236299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1200-89c5-48b1-94b2-c0fc61553acb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:19.246034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 427/873] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:18 2022] GET /identity/v3/role_assignments?user.id=d045b7cdd6024b6b82f3c4864c01e7d3&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:20.934460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da410e-0ff4-4d25-9c13-f53f58a399ea None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:20.935271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da410e-0ff4-4d25-9c13-f53f58a399ea None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:20.935505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da410e-0ff4-4d25-9c13-f53f58a399ea None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:20.938424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 447/874] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:20.948711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa0cd-40e0-42b0-b21b-23f8dff9ea63 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:20.949442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa0cd-40e0-42b0-b21b-23f8dff9ea63 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:20.949769 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa0cd-40e0-42b0-b21b-23f8dff9ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:20.983894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2e1fa0cd-40e0-42b0-b21b-23f8dff9ea63 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:20.983894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2e1fa0cd-40e0-42b0-b21b-23f8dff9ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:21.101475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 428/875] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:21.155113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518bfdc-c9b1-4f46-a64a-2832ee8ef112 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:21.155599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518bfdc-c9b1-4f46-a64a-2832ee8ef112 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:21.156138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518bfdc-c9b1-4f46-a64a-2832ee8ef112 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:21.159092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 448/876] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:21.167036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:21.167317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:21.167630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:21.198341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:21.200884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:21.312252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:21.332335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 429/877] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:21.401347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518bfdc-c9b1-4f46-a64a-2832ee8ef112 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:21.476077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:21.530884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518bfdc-c9b1-4f46-a64a-2832ee8ef112 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:21.575727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:21.577247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:21.577966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:21.578784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:21.580062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:21.582681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:21.596366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:21.598226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:21.599936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:21.608367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 449/878] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:21.613833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:21.738008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:21.781457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132c6cb-c67d-4e40-957b-fa2d6d2c9efb None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:21.837404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:21.838899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:21.839622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:21.841113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:21.843035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:21.843872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:21.847226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:21.860080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 430/879] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:21.861614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:21.880783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b1c0-2b69-4467-9e21-9f569d4517e4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:21.915401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:21.945537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:21.946564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:21.947052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:21.947526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:21.948753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:21.951372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cb7f0-423e-4d3d-a417-f8ff2fafac44 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:21.964938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:22.042896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130040-e49a-446c-a088-7f93e37d98c4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:22.075963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 450/880] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:21 2022] POST /identity/v3/services => generated 269 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:22.075963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870d3d-0f82-4b6a-b93a-acd6961ddf1c None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.075963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870d3d-0f82-4b6a-b93a-acd6961ddf1c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.075963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870d3d-0f82-4b6a-b93a-acd6961ddf1c None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.075963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 451/881] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:22.081593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36a3e-d0eb-4b7b-8bd4-b4459a20dab3 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.082315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36a3e-d0eb-4b7b-8bd4-b4459a20dab3 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.082817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36a3e-d0eb-4b7b-8bd4-b4459a20dab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.112380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2ac36a3e-d0eb-4b7b-8bd4-b4459a20dab3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:22.116129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ac36a3e-d0eb-4b7b-8bd4-b4459a20dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:22.125121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:22.126144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.126553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.126963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.128597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:22.129058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:22.131419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:22.145673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 431/882] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:22.149915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:22.242009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 452/883] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:22.282147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef7b12-0032-4630-92fd-2f05fc252803 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.282465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef7b12-0032-4630-92fd-2f05fc252803 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.282800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef7b12-0032-4630-92fd-2f05fc252803 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.285335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 453/884] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:22.291284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.291606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.291897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.319399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1774e1-d17a-49f4-bde5-8e03e2ed8cab None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:22.325570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:22.327736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:22.343755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:22.374519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:22.375613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.377324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.378071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.379211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] RBAC: Adding query filter params (interface=public, service_id=573fb9448c31499999192f59f8bd1dfa, region_id=RegionOne) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:22.379592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:22.381193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:22.391944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 432/885] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:22 2022] GET /identity/v3/endpoints?service_id=573fb9448c31499999192f59f8bd1dfa&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:22.396300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:22.446643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 454/886] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:22.552912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5fd7d-4570-48d8-af87-ce937f309125 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:22.571694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:22.597681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:22.598565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.599248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.599691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.600495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:22.602746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:22.610873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 433/887] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:22 2022] GET /identity/v3/services => generated 1268 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:59:22.629774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:22.707241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4034fd-5f09-4efe-85a2-2f501e1343b6 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:22.748657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:22.749678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:22.750137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:22.750600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:22.764967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:22.770480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:22.771509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:22.772486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:22.774971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 455/888] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:22.783109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:22.784136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:22.864864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:22.963365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b060f-8d3a-4f5c-a490-b3aec929e475 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:22.971418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:22.979493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25848d27-f62a-4c14-8f96-9f2106b5429d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.022885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.023676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.024123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.024523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.025333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:23.025760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.028675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.042784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 434/889] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:23.053160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:23.062693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.063808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.064337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.065413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.066547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.069259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:23.088413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:23.090728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:23.092116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 456/890] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:23.101001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:23.193421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6e4b-0d7d-4a2d-a077-8fab0760f7f2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.206059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:23.241767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.243353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.244133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.245096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.264962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.267443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:23.293593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:23.294622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:23.295483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:23.295483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:23.295483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:23.298143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 435/891] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:23.307893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:23.329591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4ef2b-9aa0-4128-9bd0-ed653a4eccdc None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.391378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.392083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.392361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.392633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.393282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:23.393930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.397385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.409577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 457/892] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:23.411252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:23.451228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:23.455019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d3554-205a-4c68-b55b-e8ea20903a27 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.507260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.508712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.509169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.509640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.510802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:23.511289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.513886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.531992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 436/893] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:23.541224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:23.557759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3d345-68e4-43d4-baf6-c1cc8501aafa None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.584466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:23.628334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.629066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.629566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.630079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.631306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:23.631889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.634684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.647895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 458/894] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:23.737246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2e0a9-5ee7-4f6b-840e-4c45ea590334 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:23.798940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:23.799943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:23.800221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:23.800488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:23.810669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:23.813296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:23.835223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:23.836290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:23.837231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:23.838262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 437/895] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:23.844433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:24.003596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:24.020477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c84ce-1f30-4565-a65c-8820cbce254c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:24.076875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:24.077773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:24.078195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:24.078834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:24.080083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:24.081215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:24.083639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:24.114185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 459/896] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:24.120407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:24.264466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:24.289555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e80601-9ef4-456a-8e54-e4c53f845e00 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:24.344502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:24.345754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:24.346230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:24.346664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:24.451826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:24.454772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc33f-2bea-4b8b-97bf-d7d1ee861f39 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:24.609661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:24.644101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 438/897] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:24 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:24.653907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:24.699311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0944d2-b716-48d9-ab4a-ef8f3bd85be7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:24.700221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0944d2-b716-48d9-ab4a-ef8f3bd85be7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:24.700891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0944d2-b716-48d9-ab4a-ef8f3bd85be7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:24.703799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 439/898] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:24.709571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:24.709834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:24.710133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:24.730854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:24.732195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:24.806645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ca929-925e-4ca1-8e71-397157903f97 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:24.853113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 440/899] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:24.862435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:24.863307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:24.863743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:24.864284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:24.882682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:24.885408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:24.901560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f70ebba5-7b88-4338-9df1-4811b4331ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:24.913345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:24.925842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:24.926837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:24.926837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:24.926837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:24.926837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 460/900] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:24.927090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.007333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0e22b-9139-4b62-90c0-ad132f332ed4 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.008176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0e22b-9139-4b62-90c0-ad132f332ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.008176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0e22b-9139-4b62-90c0-ad132f332ed4 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.009551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 461/901] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:25.015079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.015387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.015608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.039456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:25.041273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:25.115977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5600ec-fe0c-4cf9-9191-1fbf87446eb1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.139571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.153016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 462/902] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:25.160188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.189721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.191156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.191437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.192297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.193289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:25.193601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.196067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.215732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 441/903] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:25.223560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.310163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e2314-0331-4f80-9a5f-089875e393b9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.370215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.384216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.385890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.386897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.387378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.388950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.390899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad4c7-d456-4c11-a3a0-512240ce67e7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.391764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:25.407589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:25.409470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:25.411012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 463/904] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:25.416393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.463059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.464558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.465681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.466622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.481528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.483026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:25.498472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:25.500308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 442/905] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:25.502731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.550663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.589587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552e4ef-d131-40ce-9a25-67a2f5baf2e5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.644007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423032bb-53b0-4073-986e-9c5954a0015c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.646573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.647546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.648079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.648587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.649636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:25.650120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.652332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.666511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 464/906] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:25.667507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.686202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.686856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.687127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.687397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.688085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.690078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:25.700054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:25.701765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:25.703326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 443/907] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:25.704729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.763463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.850185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beae8e5-e9dc-42e3-a532-9bc13d8f6a65 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.864498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.875168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5186-9bf5-4647-8d2c-cf54c32bd256 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.914419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.915514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.916032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.916497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.917741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:25.918277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.921070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.921750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.922558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.922987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.923401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.924293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:25.924749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.926358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.934602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 444/908] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:25.935775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.949706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 465/909] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:25.950936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.116579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8e6ec-fd25-489b-8382-9cabe778dfe9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.128615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf12b-0982-425c-902a-ca6e2d6b68ef None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.151871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:26.160835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.161688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.177701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 445/910] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:26.178169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.178239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.178239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.178239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.178239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.178499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] RBAC: Adding query filter params (interface=public, service_id=9bfd0307d83e43b18b1c7ef41896ac63, region_id=RegionOne) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:26.178499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.178499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a56e84-e174-41e0-bbf5-d070c276518f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.191993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 466/911] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/endpoints?service_id=9bfd0307d83e43b18b1c7ef41896ac63&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:26.193848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828f74e-ab8d-44c3-a03d-fe05a5d5ee72 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.194220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828f74e-ab8d-44c3-a03d-fe05a5d5ee72 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.194618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828f74e-ab8d-44c3-a03d-fe05a5d5ee72 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.196206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 467/912] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:26.197512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828f74e-ab8d-44c3-a03d-fe05a5d5ee72 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.215311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:26.322609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2af9-92b8-4a25-863d-fa9ca4e5f9c0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.329222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828f74e-ab8d-44c3-a03d-fe05a5d5ee72 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.361512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.362124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.362388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.362655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.363384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] RBAC: Adding query filter params (role.id=818c70e959274597b8c32fe6b875d5ce, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=b86144ce73784001bc405a8e2a4b1e0d) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:26.363687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.365183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.371097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 446/913] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:26 2022] GET /identity/v3/role_assignments?user.id=b86144ce73784001bc405a8e2a4b1e0d&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=818c70e959274597b8c32fe6b875d5ce => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:26.371979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.406972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.408159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.409239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.409239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.410582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:26.410986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.414127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7725b8a-21cf-4ab4-a508-d6a4e343e0da None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.431003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 468/914] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Aug 29 18:59:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:26.431003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.431003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.431003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.463061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:26.465500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:26.535074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:26.540223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:26.558858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb332bc5-3982-4c66-a6ab-16b6c97fdfdf None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.572517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 469/915] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:26.573403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.630454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.631768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.632448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.633675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.634493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.637494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.656701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 447/916] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:26 2022] GET /identity/v3/services => generated 1268 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:59:26.736451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c5f91-74b2-4a6d-a5ce-8f41686691ac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.793348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.793982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.794259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.794524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.795205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.796760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4942-f5b6-4b07-8fa0-ba79ecfbb07a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.861195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.892397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 470/917] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:27.010334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac7721-7976-4e9c-8c9f-3d7512c909d7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:27.067112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:27.068118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:27.068522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:27.068980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:27.070049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:27.071643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94615-2128-47aa-9db6-1b4e1adf909f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:27.167559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 448/918] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:26 2022] POST /identity/v3/services => generated 241 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:29.072646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87fc1-f329-48a0-9611-6d1dbf704044 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.073219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87fc1-f329-48a0-9611-6d1dbf704044 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.073219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87fc1-f329-48a0-9611-6d1dbf704044 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.074820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 471/919] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:29.081376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46484-218f-459a-89d5-d289aacc63c5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.082232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46484-218f-459a-89d5-d289aacc63c5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.082232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46484-218f-459a-89d5-d289aacc63c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.101370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.109806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e9a46484-218f-459a-89d5-d289aacc63c5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.110654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138b4ab-4492-4139-8f74-fd309563d7cb None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.111145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138b4ab-4492-4139-8f74-fd309563d7cb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.111329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e9a46484-218f-459a-89d5-d289aacc63c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:29.111558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138b4ab-4492-4139-8f74-fd309563d7cb None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.113315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 472/920] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:29.122468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e294-5fa7-432a-8eab-a1e5b1696760 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.122897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e294-5fa7-432a-8eab-a1e5b1696760 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.123150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e294-5fa7-432a-8eab-a1e5b1696760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.156293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9bd5e294-5fa7-432a-8eab-a1e5b1696760 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.159039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9bd5e294-5fa7-432a-8eab-a1e5b1696760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:29.204674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.256157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 449/921] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:29.308028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 473/922] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:29.357689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c310e2-3ffb-4047-85ab-f4affb853f83 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.358139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c310e2-3ffb-4047-85ab-f4affb853f83 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.358615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c310e2-3ffb-4047-85ab-f4affb853f83 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.361155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 450/923] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:29.367753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9a45c-617e-4619-bbbe-00f30d0388eb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.368257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9a45c-617e-4619-bbbe-00f30d0388eb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.368741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9a45c-617e-4619-bbbe-00f30d0388eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.401349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-69c9a45c-617e-4619-bbbe-00f30d0388eb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.403500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-69c9a45c-617e-4619-bbbe-00f30d0388eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:29.522247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 474/924] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:29.618693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c310e2-3ffb-4047-85ab-f4affb853f83 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:29.682620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d983e-557a-4358-9cc9-41adc7bc2bb1 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.683569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d983e-557a-4358-9cc9-41adc7bc2bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.684356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d983e-557a-4358-9cc9-41adc7bc2bb1 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.686224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 475/925] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:29.694216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.694991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.695573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.715599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.717287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:29.797480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c310e2-3ffb-4047-85ab-f4affb853f83 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:29.806875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 476/926] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:29.808087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:29.839175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.843379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.848703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:29.851337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.851337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.851337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.862592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:29.867590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:29.869431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 451/927] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:29.871308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:29.960484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb2841-de3b-415f-bfdb-ace8459e74a2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.019730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3946c2-9fc0-4a64-ac6b-c506702908cd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.024531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.025394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.025873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.026351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.027413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.029807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:30.046061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:30.047445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.049078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 477/928] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Aug 29 18:59:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:30.052463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.094526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.095408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.095590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.096000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.097395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.100415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.104509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING keystone.server.flask.common [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 18:59:30.169316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.185076 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298e65-5653-4b97-9f84-071e1bf1cb44 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.204520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.sql.core [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:59:30.204520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:59:30.204520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [parameters: {'id': '1b0e234c388444f68b98f850077c19cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:59:30.204520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81891) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b0e234c388444f68b98f850077c19cc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b0e234c388444f68b98f850077c19cc. Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:59:30.223449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result.read() Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 18:59:30.225845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application yield resource Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise self.value Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application session.commit() Aug 29 18:59:30.227574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:59:30.229501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:59:30.231644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result.read() Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:59:30.232710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [parameters: {'id': '1b0e234c388444f68b98f850077c19cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.234040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:30.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b0e234c388444f68b98f850077c19cc. Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 452/929] 10.222.0.62 () {60 vars in 1140 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/projects => generated 262 bytes in 356 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:30.236463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.237355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.237355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.237355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.237355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.237761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:30.237761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.237761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.246451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 478/930] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:30.248154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.312136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.380580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d756f6-c644-4dc6-9a3f-db1bb84e3426 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.393880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc9db5-10fb-48c1-b54d-d1c884658e62 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.427089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.428269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.428853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.429423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.430563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:30.431173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.434169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.439470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.440722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.441003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.441310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.442326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.444788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.448189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 453/931] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:30.449554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:30.458812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:30.460422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:30.462155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.464469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 479/932] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:30.465044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.487259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.597140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59024f1-3257-47df-8c80-d987a79daf94 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.597394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d0053-aa25-4d5b-bb6d-711a68d4db94 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.612632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.639041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.639688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.639993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.640269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.643766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.644841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.645299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.645780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.654255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.655820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.656869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.657326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:30.670162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:30.671386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 454/933] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:30.672548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:30.684880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:30.686756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:30.687971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:30.687971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:30.687971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:30.687971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 480/934] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:30.687971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.777031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.813895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524f20-a9f7-4739-afe1-ff69baeb07e7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.849442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1f38-9cdd-4ef7-838c-29d43b45b859 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.866527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.867493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.868012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.868470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.869575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:30.870053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.872430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.878792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 455/935] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:30.879825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.906411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.907421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.907892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.908835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.909897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:30.910313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.912809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:30.920762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 481/936] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:30.922699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.934848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.035043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda01-4846-4e6d-9512-29567193b9d4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.070746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb15dc-f0c1-4789-a2a1-124082bf8276 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.089797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.090419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.090698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.090971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.091649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:31.091991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.093619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.111887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.119198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 456/937] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:31.120659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.121436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.122158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.122551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.122903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.123819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:31.124253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.125971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.146952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 482/938] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:31.146952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.307577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.318436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8696181-9230-4369-bb5f-a274d168aa88 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.347833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3863-2a2b-4206-882f-3b173e36ffa1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.368633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.369644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.369920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.371057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.372038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.407726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.409350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.409677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.409957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.416347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:31.416347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.416347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.422172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 483/939] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:31.423646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.492868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 457/940] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:31.500909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.514388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.604311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68f92-ca75-42d0-b5ec-210d68fe683f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.655048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.656084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.656524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.656948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.666365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.668245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:31.682945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:31.684103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:31.685057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 484/941] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:31.693627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.701529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0b4d-ad9d-46dc-8019-d04869eebc9e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.767222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.768811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.769264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.769724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.771038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] RBAC: Adding query filter params (interface=public, service_id=f39173a6bcec4376b00fe6b98de18921, region_id=RegionOne) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:31.771566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.773142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.782214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 458/942] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:31 2022] GET /identity/v3/endpoints?service_id=f39173a6bcec4376b00fe6b98de18921&interface=public®ion_id=RegionOne => generated 194 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:31.789186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:31.848408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.909005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50ed6f-80d9-4240-83eb-1a6142d75cbe None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.922829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:31.943135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad325-a665-4fbd-9db2-00340c893921 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:31.982301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:31.983385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:31.983955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:31.984463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:31.985477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:31.986014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:31.988073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:32.013379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:32.014424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:32.014912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:32.015402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:32.016338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:32.018712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:32.027844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 459/943] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:31 2022] GET /identity/v3/services => generated 1497 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:59:32.034161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 485/944] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:32.041825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:32.208213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:32.247876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa600dc-5282-4707-92b8-3142b228d1d5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:32.313296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:32.314847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:32.315326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:32.315745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:32.336959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:32.339693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:32.368130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:32.369150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:32.370067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:32.370067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:32.370067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:32.370664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 460/945] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:32.380513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:32.570856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd576de-cc86-408b-84d9-790408af9b66 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:32.635594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:32.636419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:32.636812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:32.637180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:32.637942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:32.638350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:32.639847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:32.655674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 486/946] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:32.665357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:32.836194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6f218-f690-486d-9d40-5eef79f01d7d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:32.887568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:32.894461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:32.895495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:32.895963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:32.896427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:32.904476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:32.906400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:32.927153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:32.928366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:32.929490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:32.934991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 461/947] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:32.936368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:32.987460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:33.100184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd32cdc-d119-4773-9e9f-003e7edd8837 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:33.145501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:33.146137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.146456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.146687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:33.147395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:33.147701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:33.149285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee4313-c399-48ab-ace5-a4c39bb46a23 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:33.164022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 487/948] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:33.170077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:33.342235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db802-5112-46e1-9eb5-f981d7c08d12 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:33.392177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:33.393293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.393569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.393847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/b86144ce73784001bc405a8e2a4b1e0d/roles/818c70e959274597b8c32fe6b875d5ce` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:33.396202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=b86144ce73784001bc405a8e2a4b1e0d, role_id=818c70e959274597b8c32fe6b875d5ce)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:33.397670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e0359-d324-4470-af2b-ac83eb9c034f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:33.508136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 462/949] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:33 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/b86144ce73784001bc405a8e2a4b1e0d/roles/818c70e959274597b8c32fe6b875d5ce => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:59:33.966690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1de10-7de2-437e-8080-c48137a679d0 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.968303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1de10-7de2-437e-8080-c48137a679d0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.969079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1de10-7de2-437e-8080-c48137a679d0 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:33.972968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 488/950] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:33.985733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310456d-a39c-4749-9493-588a607d46b1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.987122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310456d-a39c-4749-9493-588a607d46b1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.987615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310456d-a39c-4749-9493-588a607d46b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.016805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b310456d-a39c-4749-9493-588a607d46b1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:34.018798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b310456d-a39c-4749-9493-588a607d46b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:34.099525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:34.131755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 463/951] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:34.239031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852736-6f72-4d13-a04e-f496b4121274 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.239407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852736-6f72-4d13-a04e-f496b4121274 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.239880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852736-6f72-4d13-a04e-f496b4121274 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.242979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 489/952] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:34.251895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547127-df37-4128-987f-b677099dad9f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.252387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547127-df37-4128-987f-b677099dad9f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.252868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547127-df37-4128-987f-b677099dad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.284561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c1547127-df37-4128-987f-b677099dad9f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:34.286564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c1547127-df37-4128-987f-b677099dad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:34.403045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c65e2-940e-4c0e-9f62-07c6ef5dc15c None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.403310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c65e2-940e-4c0e-9f62-07c6ef5dc15c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.403705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c65e2-940e-4c0e-9f62-07c6ef5dc15c None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.405205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 490/953] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:34.411942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.412891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.413370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.430257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 464/954] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:34.444364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:34.446710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:34.553652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1547127-df37-4128-987f-b677099dad9f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:34.576152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 491/955] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:34.718578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1547127-df37-4128-987f-b677099dad9f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:34.742530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:34.779343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:34.780385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.780904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.781453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.782631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:34.785580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:34.811362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:34.833315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:34.850497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.sql.core [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:59:34.850497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:59:34.850497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [parameters: {'user_id': '58866e6de9a9470aad38bdabba596c2d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:59:34.850497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81891) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result.read() Aug 29 18:59:34.863305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:59:34.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application yield resource Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise self.value Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application session.commit() Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:59:34.865413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:59:34.866362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:59:34.867261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application result.read() Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:59:34.869473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [parameters: {'user_id': '58866e6de9a9470aad38bdabba596c2d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:34.871773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 18:59:34.879836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 465/956] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Aug 29 18:59:34 2022] POST /identity/v3/users => generated 160 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:34.882247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:34.952207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a2b58-4c04-4a45-90ba-068571df1489 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.006163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.007094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.007094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.007609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.008556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.010108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:35.024763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:35.032445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.033713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 492/957] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:35.034880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.052376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780e378-b58e-47b2-981c-f2680fe373c7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.105224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.105802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.106155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.106409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.118898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.121722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:35.142910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:35.145070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 466/958] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:35.148403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.172316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:35.213656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccc50a-86d2-4cae-954e-aff0fab8ae4d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.229631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:35.257569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.258212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.258500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.258787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.259374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.260795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:35.272294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:35.275156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:35.276053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 493/959] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:35.276915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.335498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff65dc-9d3d-4dab-8a46-d9545f78ecb4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.390089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.390943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.391531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.391818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.392561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] RBAC: Adding query filter params (name=compute) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:35.392870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.394498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.409757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 467/960] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:35.410058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.433057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f265a7-907d-4671-863f-e9c2d6491c0a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.475984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.477040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.477529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.477808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.479010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:35.479923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.481660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.525503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:35.530730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4cf4-29fa-486b-8286-d3b4cf3a5dfe None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.532893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 494/961] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:35.532893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.535964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:35.577672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.578392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.578670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.578995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.580406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:35.580692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.583832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.589390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 468/962] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:35.599398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.672989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc9a5-b868-4139-99e1-ada956bd214d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.716331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.717387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.718028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.718722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.721239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:35.721994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.724949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.730865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f385c8-3383-45a0-8b36-119a09a1c172 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.734739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 495/963] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:35.780753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:35.781788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.782218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.782650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.783662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:35.784408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:35.786203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401de4e-80f3-45d8-b0af-fa7ea79f70cf None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:35.792805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 469/964] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:35.799756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:35.906363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:35.954502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92104-827b-4bc6-ae9c-5dfa60d017e9 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:35.967686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e2847-aa98-4a00-998f-964072f08d51 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.968037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e2847-aa98-4a00-998f-964072f08d51 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.968322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e2847-aa98-4a00-998f-964072f08d51 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.970203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 470/965] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:35.976862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:35.977111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:35.977415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:35.996705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:35.998468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:36.006179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:36.006895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.007243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.007582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.008383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:36.009817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:36.029815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:36.087770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 471/966] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:36.108197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 496/967] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:36.381419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:36.506071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3700ad-457d-4116-acd1-42cee87429c2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:36.552416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553322e9-869d-499c-9169-f427f5943589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:36.553266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553322e9-869d-499c-9169-f427f5943589 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.553544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553322e9-869d-499c-9169-f427f5943589 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.553814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553322e9-869d-499c-9169-f427f5943589 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.566605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553322e9-869d-499c-9169-f427f5943589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-553322e9-869d-499c-9169-f427f5943589 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:36.571488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:36.572662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:36.573833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 472/968] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:36.581323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:36.703482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c743b-467d-449b-9e07-7a3e33ca3e16 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:36.729494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:36.749557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:36.750499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.750833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.751101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.752530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:36.753014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:36.755381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:36.778288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 497/969] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:36.780012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553322e9-869d-499c-9169-f427f5943589 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:36.797694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:36.895680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553322e9-869d-499c-9169-f427f5943589 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:36.936863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:36.937976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.937976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.938414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.950194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:36.952476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:36.968566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:36.969725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:36.971317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:36.971317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:36.971317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:36.978148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 473/970] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:36.979467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:37.096985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3082c1-f4c4-4978-bfe2-a64c73d4e3bb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:37.138452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:37.139271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.139648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.140061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.140833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:37.141236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:37.142763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:37.155541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 498/971] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:37.161488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:37.245467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:37.302071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77243aca-38b6-4ab4-83ae-af21d3efc3bc None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:37.341407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:37.341803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.342255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.342704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.355374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:37.356350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:37.370781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:37.371786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:59:37.372827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:37.376246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 474/972] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 18:59:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:37.378467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:37.447704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:37.517980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb28c37-6d8c-4d5c-9ace-6acd2bd59edf None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:37.559139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:37.560158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.560631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.561070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.562154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:37.562537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:37.568229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dae1f4-4530-4566-9a55-8a7e6e9da987 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:37.594310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 499/973] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Aug 29 18:59:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:37.597916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:37.733699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763f59a-53c7-4e75-a3cd-b473367091c9 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:37.763702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:37.781501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:37.781501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.781501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.781501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.837052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2661-9e5c-4c21-8c34-5c2f84608eca None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.837857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2661-9e5c-4c21-8c34-5c2f84608eca None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.839223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2661-9e5c-4c21-8c34-5c2f84608eca None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.841554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 500/974] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:37.846746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.847048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.847675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.866933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:37.868156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:37.880060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:37.881545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770cbdf-6ba9-4829-aca4-14231a5a18a4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:37.951972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 501/975] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:38.015821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 475/976] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:38.025290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.060610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.115168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9889f-aa1a-40b0-9fa5-d5719fc74779 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.115453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9889f-aa1a-40b0-9fa5-d5719fc74779 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.115737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9889f-aa1a-40b0-9fa5-d5719fc74779 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.117269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 476/977] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:38.122431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.122712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.122983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.136437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a720332-fc33-4016-b6df-2112c73ecf85 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.142031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:38.143985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:38.163704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.173432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.176157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.176157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.176157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.185434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.186883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:38.202469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:38.204051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:38.205809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:38.205809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:38.205809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:38.205809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 502/978] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:38.209774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.229366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 477/979] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:38.230234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.306298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.326590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adde88-463c-4af9-8b7e-2ed52ad8eaae None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.346589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e52116-b4ba-4498-8540-81ceb67cf6e1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.366015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.366874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.367447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.367721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.368395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:38.368700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.370188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.383357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.383560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 503/980] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:38.384658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.385138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.385612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.386507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.388288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.388953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.494910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7b94-3cbd-4bb6-b19a-bb25a033e940 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.528635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.529262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.529533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.529797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.537115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.538537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.548182 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 478/981] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:38 2022] POST /identity/v3/services => generated 242 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:38.551774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:38.551921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:38.553165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:38.554431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 504/982] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:38.561211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.565636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.575556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.686170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f9ee5-fa48-4b3c-8173-6459aa95d103 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.690179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70e58-c8c5-4134-b79a-3693ca221b58 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.728399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.729047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.729329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.729596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.730181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.731565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.732103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.732717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.732999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.733259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.734018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:38.734322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.735751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:38.742994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:38.744178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:38.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 479/983] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:38.750882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 505/984] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:38.751514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.757672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.826515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.841374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.870343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d047-d04d-404a-aa61-a3dca20d6673 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.872181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-e149-452a-b8ba-b21f3f1612c9 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.912391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.912627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.913006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.913110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.913275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.913370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.913546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.913641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.914169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:38.914278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] RBAC: Adding query filter params (role.id=818c70e959274597b8c32fe6b875d5ce, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=b86144ce73784001bc405a8e2a4b1e0d) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:38.914467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.914619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.915831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.916041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.922060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 480/985] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:38.927891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 506/986] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/role_assignments?user.id=b86144ce73784001bc405a8e2a4b1e0d&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=818c70e959274597b8c32fe6b875d5ce => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:38.928094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:39.061331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae291-5d4d-45d7-a9c8-6c63e43eb111 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:39.102273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:39.102889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.103166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.103437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.104141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:39.104457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:39.105882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:39.111390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 481/987] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:39.117210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:39.234512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1191371-47ca-46d3-af05-b9b2c058f644 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:39.264686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:39.274489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:39.275108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.275381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.275650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.276392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] RBAC: Adding query filter params (interface=admin, service_id=f39173a6bcec4376b00fe6b98de18921, region_id=RegionOne) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:39.276698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:39.278084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278e05a-f6ab-40f4-9534-34361a99c2dc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:39.288402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 507/988] 10.222.0.62 () {56 vars in 1249 bytes} [Mon Aug 29 18:59:39 2022] GET /identity/v3/endpoints?service_id=f39173a6bcec4376b00fe6b98de18921&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:39.289961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:39.292860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:39.399933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59220-7e7b-4e5e-a639-648b6c81658b None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:39.434119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:39.436234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.436234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.436234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.436234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:39.437304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f4fd0-3a74-409e-8fe7-7257215d60aa None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:39.451883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 482/989] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:39 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:59:40.538755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073dd4-996f-4e80-b86e-3dfce6b65b13 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.539345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073dd4-996f-4e80-b86e-3dfce6b65b13 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.539345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073dd4-996f-4e80-b86e-3dfce6b65b13 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.540900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 508/990] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:40.546809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd37-c626-4b02-b1d0-e2dac0d866f1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.547337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd37-c626-4b02-b1d0-e2dac0d866f1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.547337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd37-c626-4b02-b1d0-e2dac0d866f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.566841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0541dd37-c626-4b02-b1d0-e2dac0d866f1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:40.568257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0541dd37-c626-4b02-b1d0-e2dac0d866f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:40.634103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:40.648981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 483/991] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:40.904758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aabf03-c609-4b3e-9a13-7addbe1c648d None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.904948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aabf03-c609-4b3e-9a13-7addbe1c648d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.905210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aabf03-c609-4b3e-9a13-7addbe1c648d None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.906676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 509/992] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:40.912457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.912817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.913292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.917397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aabf03-c609-4b3e-9a13-7addbe1c648d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:40.932407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:40.933601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:41.016206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 484/993] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:41.038800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aabf03-c609-4b3e-9a13-7addbe1c648d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.073742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.074371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.074649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.074922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.075552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.076953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:41.087257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:41.088301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:41.092838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 510/994] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:41.095207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.202870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d905ca-27a6-41ae-bf31-1b8a0b242843 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.229672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.238732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.239337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.239613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.241313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.241313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:41.241313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.242167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.246999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 485/995] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:41.254382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.284770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.317578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.364760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f44f-d4e9-4fd1-8669-e289fba067fb None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.397671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816ec8f-e701-4929-9539-c8a992de0012 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.400988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.401627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.401879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.402149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.402779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:41.403079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.404498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1993b-8d2e-45e9-a25c-b65bc496c504 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.410716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 511/996] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:41.412062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db3b7-05c0-4b35-9f28-0ad1cfaf313c None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.412346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db3b7-05c0-4b35-9f28-0ad1cfaf313c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.412622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db3b7-05c0-4b35-9f28-0ad1cfaf313c None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.414089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 512/997] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:41.416273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db3b7-05c0-4b35-9f28-0ad1cfaf313c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.435913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.437664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.437664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.437664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] PATH_INFO: `/v3/services/image` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.437961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.439675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-5b9ced31-347b-4a18-ac03-a5838839040e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:41.448860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:41.450164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:41.451101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 486/998] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:41.451971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.451971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.451971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.471289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:41.473207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:41.534424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.535535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db3b7-05c0-4b35-9f28-0ad1cfaf313c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.554301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 487/999] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:41.554301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.558493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.575833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.576529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.576898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.577269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.578039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] RBAC: Adding query filter params (interface=public, service_id=1c7f557801644576a7b107b89201f5df, region_id=RegionOne) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:41.578414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.580000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.585912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 513/1000] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/endpoints?service_id=1c7f557801644576a7b107b89201f5df&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:41.591982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.671157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f390be3-af12-4376-8662-f5700daa06c7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.711180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2a14-775b-4c41-a2d2-b3c9bf406411 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.717386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.718515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.718863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.719200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.720816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:41.721456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.723290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.729378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 488/1001] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:41.734695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.748425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.749043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.749316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.749593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.750181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.751644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.758230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 514/1002] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:59:41.792007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.825189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.854684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345193ed-49dd-4c38-bc69-ee5c629d5ad0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.884219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.901473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.902087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.902359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.902619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.903650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:41.903912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.905332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.910772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 489/1003] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:41.950191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35edb4-dbfe-488d-8dbf-da7f41100146 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.993544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.994312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.994730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.995102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.995783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.997286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:42.008958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:42.010297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:59:42.011346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:42.013255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 515/1004] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:42.019782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:42.127698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02759c-3adc-4987-a335-ccaa38d61be3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:42.141324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:42.163485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:42.164629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:42.165252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:42.165972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:42.166607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:42.166908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:42.168311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:42.174881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 490/1005] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:42.179473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:42.281082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861285-b0cc-4af3-b35c-cdc7afc55c35 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:42.294511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:42.318398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:42.319090 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:42.319373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:42.319649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:42.320318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:42.320622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:42.322046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccca40-a77e-4e06-9744-ea4c90a739f6 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:42.331883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 516/1006] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Aug 29 18:59:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:42.333674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:42.441035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3362ce-1fce-4779-8397-e8d8b1fb6b51 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:42.477805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:42.478408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:42.478682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:42.478951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:42.479618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:42.481077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b93fe-ce05-4098-bc4b-7abe4b3f9d88 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:42.598272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 491/1007] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:43.733380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5b0af-149f-4e22-8244-1f8412c9984c None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:43.733900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5b0af-149f-4e22-8244-1f8412c9984c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:43.733900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5b0af-149f-4e22-8244-1f8412c9984c None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:43.735296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 517/1008] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:43.740963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012fba-fc21-4d08-8eac-86a49d9d041d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:43.741351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012fba-fc21-4d08-8eac-86a49d9d041d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:43.741530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012fba-fc21-4d08-8eac-86a49d9d041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:43.759790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ef012fba-fc21-4d08-8eac-86a49d9d041d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:43.761029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ef012fba-fc21-4d08-8eac-86a49d9d041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:43.791227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:43.835913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 492/1009] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:43.862413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f5da3-5653-4aaf-8716-446c4aeb647d None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:43.862696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f5da3-5653-4aaf-8716-446c4aeb647d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:43.862950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f5da3-5653-4aaf-8716-446c4aeb647d None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:43.864454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 518/1010] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:43.869389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:43.869683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:43.869955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:43.888977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:43.890163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:43.957804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 493/1011] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:44.111762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f5da3-5653-4aaf-8716-446c4aeb647d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.218794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f5da3-5653-4aaf-8716-446c4aeb647d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.227664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.256578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.257144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.257423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.257695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.258274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:44.259621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:44.269260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:44.270279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:44.271100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 519/1012] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:44.276097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.320968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:44.325412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:44.334384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f22fe-8d04-4c73-86bc-75e2e334b1aa None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.368136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.368740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.369013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.369414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.370186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:44.371555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a468-d554-4f44-b7db-8311fb5ad8f4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:44.383282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91cf77-dc4a-4a6e-9f92-b9d8d4263748 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.422858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.423579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.423883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.424150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.424790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] RBAC: Adding query filter params (name=compute) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:44.425202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:44.426568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:44.431349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 520/1013] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:44.437299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.462149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 494/1014] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:44 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:59:44.553936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4def4-8a8a-4a2a-978a-ce428b991abf None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.556427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:44.584832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43470-00e6-4f35-8843-21c2fba6e3df None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.585232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43470-00e6-4f35-8843-21c2fba6e3df None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.585563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43470-00e6-4f35-8843-21c2fba6e3df None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.587100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 495/1015] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:44.592841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.593128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.593408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.601731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.602343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.602615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.602879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.603498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:44.603794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:44.605182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:44.613471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 521/1016] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Aug 29 18:59:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:44.615399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:44.617191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:44.619277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.694058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 496/1017] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:44.730955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fceec-a670-4a89-a285-cfa9b71978e0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.765361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.765996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.766253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.766489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.767731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:44.769177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:44.871970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 522/1018] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:59:44.968957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:45.061646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:45.081687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237e55f-ebcb-467d-82bd-7b3edf219453 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:45.123716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:45.124675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:45.125136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:45.125402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:45.127043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:45.128517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba6c49-c7bd-4f0d-b6d0-2f09b8c3bf46 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:45.130247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 497/1019] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:45.136074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:45.161276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:45.244786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b63ca-41be-4a38-88fd-b71b896c8cac None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:45.284422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:45.285031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:45.285304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:45.285573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:45.286220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:45.287634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c557da-0999-4e2b-a92f-87f3acc236c0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:45.392293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 523/1020] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 18:59:45 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:59:46.421839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb092a8c-2541-4d1f-942f-91c12998f0e2 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:46.422347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb092a8c-2541-4d1f-942f-91c12998f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:46.422347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb092a8c-2541-4d1f-942f-91c12998f0e2 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:46.424157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 498/1021] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:46.429295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6d0f-495e-4454-a948-82ce6f252c9b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:46.429726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6d0f-495e-4454-a948-82ce6f252c9b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:46.429897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6d0f-495e-4454-a948-82ce6f252c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:46.447798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7faa6d0f-495e-4454-a948-82ce6f252c9b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:46.449014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7faa6d0f-495e-4454-a948-82ce6f252c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:46.524912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 524/1022] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:46.792564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb092a8c-2541-4d1f-942f-91c12998f0e2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:46.896427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb092a8c-2541-4d1f-942f-91c12998f0e2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:46.914482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312732-63be-47fe-9027-3bd464f221b2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:46.914804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312732-63be-47fe-9027-3bd464f221b2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:46.915038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312732-63be-47fe-9027-3bd464f221b2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:46.916544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 525/1023] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:46.921292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:46.921595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:46.921848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:46.928971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:46.934962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:46.935685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:46.936049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:46.941036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] PATH_INFO: `/v3/services/image` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:46.941036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:46.941036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:46.941487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:46.942732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:46.948171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:46.949167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:46.950008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:46.951986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 499/1024] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Aug 29 18:59:46 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:46.958376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:46.974254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.026488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 526/1025] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:47.072435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbb0b-b3bf-4e56-a608-d06161183b0c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.108189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.108797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.109073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.109346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.109964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] RBAC: Adding query filter params (name=image) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:47.110263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.111833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.121526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 500/1026] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:47.122628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.224876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadd61d-e686-4074-ab5c-755550e3a793 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.254595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.259994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.260599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.260870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.261141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.261766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] RBAC: Adding query filter params (type=image) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:47.262060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.263470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.269315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 527/1027] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:47.275013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.296379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.364271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.389463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269556b8-88fa-43fc-bc67-5856f9fabd38 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.408723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ab0f7-d4df-43a1-ab04-34f5cf131b2e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.422243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.422886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.423189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.423460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.424188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] RBAC: Adding query filter params (interface=public, service_id=c788f99a087748c08a36db5f70ad1f58, region_id=RegionOne) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:47.424492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.425858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356484-e665-4f3d-afc0-63bed367d302 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.431102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 501/1028] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/endpoints?service_id=c788f99a087748c08a36db5f70ad1f58&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:47.432281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a81a-c4d1-47bd-922c-464a7b09a921 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.432584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a81a-c4d1-47bd-922c-464a7b09a921 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.432849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a81a-c4d1-47bd-922c-464a7b09a921 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.434034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 502/1029] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:47.436527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09a81a-c4d1-47bd-922c-464a7b09a921 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.449006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.449639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.449917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.450189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.461529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-7a2d72f9-c490-4928-9fe8-38d45cc8ae54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:47.466380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:47.467458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:47.468355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 528/1030] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:47.469063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.469705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.469985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.487341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:47.488570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:47.540121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.545264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09a81a-c4d1-47bd-922c-464a7b09a921 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.559898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 529/1031] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:47.559898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.582324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.582948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.583190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.583454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.584157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.585464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.603888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 503/1032] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:59:47.676246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257adf2c-aac9-403d-a8d2-5f1d38d7e9f4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.712647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.713271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.713543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.713811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.714480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:47.714846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.716268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.726261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 530/1033] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:47.732805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.736310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.835036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.842032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c80ce-d749-438f-b942-e43d6682aa23 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.879804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.880469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.880747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.881019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.887763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.892554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.893772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:47.908597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:47.909595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:47.910498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:47.910498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:47.910498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:47.914697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 504/1034] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:47.915622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.945719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10693-fc6a-48e8-974b-6a1c2be482b9 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.981518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.992504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.993155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.995950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.995950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.995950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:47.996296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:47.997918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 531/1035] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:48.003973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:48.034127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6271f-d6b4-434c-a623-180e7c7a47e1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.070876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.071479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.071759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.072061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.072703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:48.073008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.074376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.086936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 505/1036] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:48.092401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:48.115053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc333c3-0e2c-4c34-8239-f40bbfeeec97 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.133560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:48.152682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.153295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.153570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.153843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.154500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.155978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.199696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246ca-65fc-4208-96b9-294028ef4be2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.213507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:48.234151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.234778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.235096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.235368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.243139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.244585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:48.257349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:48.258297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:48.259869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 506/1037] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:59:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:48.264231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:48.272389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 532/1038] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 18:59:48 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:59:48.384363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1fc1d-cd82-4724-9e6d-123076eda7b1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.425407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.426032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.427113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.427113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.427303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:48.427605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.429094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.447776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 507/1039] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:59:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:48.453545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:48.572622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641aec3-6f0a-49f0-8738-9f2cea72826d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.608968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.609575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.609851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.610120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.653272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:48.680787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.682268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.790269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 533/1040] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:48.791902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:48.825839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:48.898164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744366-0d05-4d08-b602-48e6fb58502e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:48.932761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:48.933411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:48.933690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:48.933960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:48.944810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.946368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:48.962290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:48.963310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:48.964230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:48.964230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:48.964230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:48.964230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 508/1041] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:48.969336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:49.050819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:49.079264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ded5a-42c3-437f-be83-83ea4fb84b13 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:49.114890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:49.115503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.115779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.116081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.116719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:49.117018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:49.118378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:49.130686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 534/1042] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:49.136315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:49.221589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:49.238628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298321a-6ccb-4132-b427-ea27d9c6433f None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:49.273441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:49.274210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.274592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.274959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.283370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:49.284847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:49.296385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:49.297519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:49.299216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:49.302538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 509/1043] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:49.309019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:49.413262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b6655-98ae-4550-98d2-62c249e88209 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:49.443681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:49.449199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:49.449997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.450388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.450755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.451633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:49.452072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:49.453650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408635-1376-4cc5-a3d0-5705f0a25452 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:49.466951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 535/1044] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:49.473586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:49.536884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60702-c19a-49e6-9f51-e26d901fe707 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.536884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60702-c19a-49e6-9f51-e26d901fe707 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.536884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60702-c19a-49e6-9f51-e26d901fe707 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.538235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 536/1045] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:49.543067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.543346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.543625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.562434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:49.563575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:49.587509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8299fb-3a70-4c9c-9c8a-bd3f1e5b2859 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:49.625155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:49.625752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:49.626055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:49.626320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:49.627136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] RBAC: Adding query filter params (role.id=818c70e959274597b8c32fe6b875d5ce, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:49.627535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:49.629105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:49.632900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 537/1046] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:49.639666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 510/1047] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:49 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=818c70e959274597b8c32fe6b875d5ce => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:49.902302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.012271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2ea85-4469-405e-817a-c2cc11ec5173 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.036516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.048086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.048721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.048997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.055282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.055282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.055282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:50.060154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:50.061140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:50.061970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 538/1048] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Aug 29 18:59:49 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:50.067494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.080324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.171084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c3d8-c724-492d-8bda-3c649dca513a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.204715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.205327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.205596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.205864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.206479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:50.206779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.208169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b40f-0f5e-497a-a713-3aab36ff8af0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.218024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 511/1049] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:50.219047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.320615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3429e-cd5a-4422-9275-942da1bfca76 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.321265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3429e-cd5a-4422-9275-942da1bfca76 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.321373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07a244-2988-46a6-9498-5e1bfedd12d5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.321831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3429e-cd5a-4422-9275-942da1bfca76 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.323355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 512/1050] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:50.328519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.329995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.330233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.348776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:50.350208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:50.357099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.357704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.357983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.358252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.358872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] RBAC: Adding query filter params (type=image) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:50.359173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.360573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.365836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 539/1051] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:50.371503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.431976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 513/1052] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:50.432936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.477798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c188-648c-4032-b413-4e7f79d84b4a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.510820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.511424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.511698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.513576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.513576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.514168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.587705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 540/1053] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 18:59:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:59:50.703232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.705529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.805202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa20479-d850-44ae-8002-c07900297c02 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.846010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.846634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.846913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.847185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.857727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:50.863089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:50.864257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:50.865143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 514/1054] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:50.870108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.979183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755c0f-03ff-42a1-b9b9-1c64382856e4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.989186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:51.018691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:51.019304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.019581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.019866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.020548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:51.020882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:51.022213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:51.035886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 541/1055] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:51.038848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:51.101768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:51.145817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c78d-fe0d-4993-8b16-415def34dfd6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:51.179613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:51.180280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.180555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.180923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] PATH_INFO: `/v3/users/05853d8ba104456aafd07c577bf0eafc` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.195021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] RBAC: Authorizing `identity:get_user(user_id=05853d8ba104456aafd07c577bf0eafc)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:51.196523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:51.198327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 515/1056] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/users/05853d8ba104456aafd07c577bf0eafc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:51.204725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:51.309234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d8370-8cb3-4519-8d04-56d79b330cad None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:51.346421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:51.347022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.347294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.347564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.382494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:51.415714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:51.417195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28102cb7-baa5-4a67-8a40-8f32a3164b34 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:51.522140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 542/1057] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:51.527680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:51.539234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae96c09-31d2-47d5-ad39-94b201f91652 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.539541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae96c09-31d2-47d5-ad39-94b201f91652 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.539788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae96c09-31d2-47d5-ad39-94b201f91652 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.541262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 543/1058] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:51.545922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.546203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.546473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.565512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:51.566625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:51.632866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:51.638705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 544/1059] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:51.643738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709774c6-9048-4fd3-94e1-c340ba7db64c None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:51.662510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:51.685846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:51.686510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.686782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.687047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.688229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:51.690277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:51.696309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 516/1060] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:51.698392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:51.803612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9952-8dc0-4422-a10d-27ff42bd2157 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:51.837348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:51.837885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.838164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.838434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.839154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009, user.id=05853d8ba104456aafd07c577bf0eafc) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:51.839462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:51.840909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:51.852104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 545/1061] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/role_assignments?user.id=05853d8ba104456aafd07c577bf0eafc&scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:51.906306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.027518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66fcf3-9b3b-4d0d-b12c-b8f65103fd19 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.065742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.066384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.066663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.066935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.077676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:52.082637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:52.083575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:52.084460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 517/1062] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:52.091503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.155314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:52.196011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c82e3-718d-4525-a1a7-878ce5493419 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.231199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.231864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.232155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.232429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.233095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:52.233422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.234913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.243813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 546/1063] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:52.249296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.301070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:52.358020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208eff5-2166-4cf0-88f3-075890947c48 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.394060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.394876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.395159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.395427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.405864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.407349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-bde998a8-13e4-419d-bfc9-c69663c63bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:52.422168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:52.423666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:52.424658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:52.424658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:52.424658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:52.424658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 518/1064] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:52.430951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.537168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d9270-9e21-420d-8f22-7672f8d72a8d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.554282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6c6e6-8427-4114-a127-71f920227f0c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.554556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6c6e6-8427-4114-a127-71f920227f0c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.554833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6c6e6-8427-4114-a127-71f920227f0c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.556192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 519/1065] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:52.560815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.561113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.561359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.565358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:52.578119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.578893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.579173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.579934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:52.580061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.580236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:52.580525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.580619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:52.581889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.595984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 547/1066] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:52.601665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.654017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 520/1067] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:52.700765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750b79b-2be4-4497-86cc-0b99a8ecb517 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.735288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.736595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.736709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.736972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.744189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.745616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:52.756986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:52.757926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:52.758958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:52.762821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 548/1068] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:52.768070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.793823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:52.868904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834efa0-627a-4fac-8156-c8b0b287feca None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.904416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.904951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.905284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.905557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.906323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:52.906666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.908856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.927053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 521/1069] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:52.928517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.932776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.983121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.025584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.042146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e8d3c-4e46-498d-9133-ead024bc14e8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.055274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d176-f96c-4159-9d37-fddd700c79a8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.078321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.079506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.079837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.080133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.090673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.091546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.092137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.092260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.092505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.092737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:53.107937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:53.108904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:53.109801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:53.109801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:53.109801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:53.109801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 522/1070] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:53.114760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.170227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.172198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.218339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665840e5-c235-48b2-8fe9-07630bd1b772 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.224856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.254433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.255038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.255314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.255580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.256249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:53.256545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.257917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.260346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.271089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 523/1071] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:53.276712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.297510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 549/1072] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:53.298752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.395585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d33ec-7067-4f2b-8474-bd6f67e3883a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.415346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f3600-eee3-4037-abb7-6189736d6f81 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.435223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.436092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.436551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.437019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.444319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.445778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.455607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.456554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.457032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.457526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:53.457795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:53.458710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:53.459683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 524/1073] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:53.464619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.475091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.477005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.480187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:53.495375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:53.496581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:53.497740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:53.497740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:53.497740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:53.499587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 550/1074] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:53.505162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.528489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.577684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f3f8-b80c-49b1-8026-f6665f1fe791 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.608847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6713d-86b3-4500-974b-3d4da6e84741 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.615083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.615695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.616000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.616274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.617286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:53.617286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.618682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.631625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 525/1075] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:53.637249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.650609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.651221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.651496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.651765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.652436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:53.652737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.654126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0cd9-bf94-4452-aa7d-ea9429fa1570 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.666661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 551/1076] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:53.768381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fee4bc-38ff-40ce-8b9b-a459e0e79b2e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.786677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521789cc-d2be-4f04-afce-f2274ce2419f None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.786961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521789cc-d2be-4f04-afce-f2274ce2419f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.787231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521789cc-d2be-4f04-afce-f2274ce2419f None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.788824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 552/1077] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:53.788952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.794473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.794750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.795019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.808408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.809080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.809353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.809621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/818c70e959274597b8c32fe6b875d5ce` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.811807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=bf0af6b468c7442e818efec1543b7eef, role_id=818c70e959274597b8c32fe6b875d5ce)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:53.813339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:53.814313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:53.815576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:53.870835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.887482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 553/1078] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:53.914370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 526/1079] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:53 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/bf0af6b468c7442e818efec1543b7eef/roles/818c70e959274597b8c32fe6b875d5ce => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:59:54.160041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:54.258513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bab7a8-698f-411f-b92d-34f1ffb545a4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:54.291869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:54.292509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:54.292786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:54.293057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:54.303062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:54.307955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:54.308968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:54.309804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 554/1080] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:54.316037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:54.415329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3f36-c82f-4f09-8518-f385ed1a2a97 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:54.449748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:54.450339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:54.450610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:54.450876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:54.451519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:54.451815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:54.453323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:54.461501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 527/1081] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:54.467887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:54.516232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:54.577249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04a18-6c9b-4a90-840f-ff36e484e02c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:54.615336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:54.616751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:54.617168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:54.617580 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] PATH_INFO: `/v3/users/05853d8ba104456aafd07c577bf0eafc` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:54.618842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] RBAC: Authorizing `identity:get_user(user_id=05853d8ba104456aafd07c577bf0eafc)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:54.620512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:54.627785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 555/1082] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:59:54 2022] GET /identity/v3/users/05853d8ba104456aafd07c577bf0eafc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:54.629310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:54.670086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:54.734506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84f18-bcf0-4ccd-95d4-3c43ff0ce4b0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:54.771595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:54.772257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:54.772528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:54.772804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:54.773944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:54.775382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a8af3-6c10-44ba-bb59-afcb080f7144 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:54.777250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 528/1083] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:59:54 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:54.783210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:54.893166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de5e2-4eef-41e5-9520-0fb3fbc7233b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:54.908189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:54.929375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:54.929963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:54.930239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:54.930509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009/users/05853d8ba104456aafd07c577bf0eafc/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:54.932418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] RBAC: Authorizing `identity:create_grant(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009, user_id=05853d8ba104456aafd07c577bf0eafc, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:54.933850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636052e-a1f4-4746-a534-56c75819277d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:55.003628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 556/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 18:59:54 2022] PUT /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009/users/05853d8ba104456aafd07c577bf0eafc/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:59:55.837474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95f15-32a1-4ad1-98b1-173fb29ef4c7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:55.838049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95f15-32a1-4ad1-98b1-173fb29ef4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:55.838049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95f15-32a1-4ad1-98b1-173fb29ef4c7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:55.839354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 529/1085] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:55.845173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:55.845600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:55.845694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:55.864212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:55.865388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:55.930946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 557/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:59:56.196219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e95f15-32a1-4ad1-98b1-173fb29ef4c7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.294029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e95f15-32a1-4ad1-98b1-173fb29ef4c7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.300996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.327449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.328099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.328407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.328676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.339130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:56.347446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:56.348502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:56.349651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 530/1087] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:56.352242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.448419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259dab6-3854-4df9-9a37-4c9e75f21bfb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.450620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.482086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.482701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.483001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.483274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.483965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:56.484265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.485654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.494418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 558/1088] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:56.500071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.600408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a1c0b-449a-477e-9b32-7e825063ffac None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.633695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.634312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.634592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.634862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.645271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.646739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:56.660668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:56.666437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:56.667448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:56.667448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:56.667448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:56.667448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 531/1089] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:56.667739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.766145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34eda-213c-4644-91d1-b7c624026586 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.798669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.799218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.799488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.799750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.800408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:56.800705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.802064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3152244-512e-43f2-9012-b3a3b466413b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.814167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 559/1090] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:56.819770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.840065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.901486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe35b-8a9b-4a65-8e5f-7feb95dc4d6b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.902027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe35b-8a9b-4a65-8e5f-7feb95dc4d6b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.902027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe35b-8a9b-4a65-8e5f-7feb95dc4d6b None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.903329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 560/1091] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:56.908302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.908582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.908851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.921833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7090ccd-c8cf-457c-a876-eba5382234f1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.926769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:56.927971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:56.948682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.957513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.958142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.963606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.963606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.966084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.967566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:59:56.979930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:56.980987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:56.981957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 532/1092] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:56.987381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.999213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 561/1093] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:59:57.066717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.088284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752233d7-3487-43b4-9ce6-8cb9c3659256 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.122133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.122724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.122996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.123263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.124000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:57.124296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.125722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.141998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 533/1094] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:57.147787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.244283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cda31b-3f8a-485b-bf33-7790b6bd2a7c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.267251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.277843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.278446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.278716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.278983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.304657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.347507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.349163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.366021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf10eb-0509-434a-bfd5-63d7a9575aa0 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.401697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.402331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.402605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.402874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.405470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.414392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:59:57.419357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:57.420406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 18:59:57.421246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 534/1095] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:57.427152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.452899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 562/1096] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:59:57.459226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.512059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.555139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5eaa-d4b1-4466-880a-d324a4aba0db None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.575038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dec9d-18f9-465b-b8c9-a1d6f77b7371 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.589782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.590118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.590390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.590654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.601385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.602748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.611402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.612041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.612319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.612592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.613244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:57.613545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.614920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:59:57.616788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:57.617898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:59:57.619012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:59:57.619012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:59:57.619012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:57.619012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 563/1097] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:59:57.623554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.625805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 535/1098] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:57.631382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.689540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.725109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd5d2-9858-4048-a127-ec877e06f5f6 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.732581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4d98-a439-404c-81cd-dcee96b58719 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.733797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.759145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.759758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.760065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.760340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.760980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:57.761285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.762686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.765864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.767062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.767349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.767596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] PATH_INFO: `/v3/users/05853d8ba104456aafd07c577bf0eafc` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.768750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] RBAC: Authorizing `identity:get_user(user_id=05853d8ba104456aafd07c577bf0eafc)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.770155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.771730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 536/1099] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/users/05853d8ba104456aafd07c577bf0eafc => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:57.777462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.783516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 564/1100] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:57.784568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.874457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45835e-399b-4d0b-90be-2ca5be7b34d8 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.880592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a9254-567f-4b67-a91b-c3f66f27cfc6 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:57.899707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.909396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.909995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.910264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.910533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.914610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:57.915071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:57.915173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:57.915398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:57.921990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.923481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:59:57.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:59:57.936650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 18:59:57.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 565/1101] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:59:57.942994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:57.971498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:57.984983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:57.990005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.043214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b6f-5620-46a2-aa2d-7f7dce11f4e4 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:58.077989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:58.078806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.079581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.079883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.080688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:58.080990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:58.082417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.088291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 537/1102] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:59:58.094043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:58.104030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 566/1103] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 18:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:58.105262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:58.120264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:58.193591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:58.193833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43c727-7818-482f-a454-aa052ea9c307 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:58.205212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56df4-ff54-480a-8e32-df4600ebdc7f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:58.228129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:58.228759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.229039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.229313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.230413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:58.231842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.233727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 538/1104] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 18:59:58 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:58.240141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:58.244009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:58.244630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.244903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.245169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.245882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] RBAC: Adding query filter params (role.id=818c70e959274597b8c32fe6b875d5ce, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=bf0af6b468c7442e818efec1543b7eef) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:58.246182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:58.247673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ae577-760d-4a6f-9ec8-ad49ffcea3cf None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.254541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 567/1105] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:58 2022] GET /identity/v3/role_assignments?user.id=bf0af6b468c7442e818efec1543b7eef&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=818c70e959274597b8c32fe6b875d5ce => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:59:58.341449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:58.348127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44a849-c446-4a48-b4f4-80b70bcf9905 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:58.385580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:58.386230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.386508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.386777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.387499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009, user.id=05853d8ba104456aafd07c577bf0eafc) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:59:58.387814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:58.389310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbdbe1-985f-478c-9cd0-2bf9b12cc1a4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.399260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 539/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 18:59:58 2022] GET /identity/v3/role_assignments?user.id=05853d8ba104456aafd07c577bf0eafc&scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:00:00.350021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9847a6-3655-4854-91ca-1c97c5856881 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:00.350564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9847a6-3655-4854-91ca-1c97c5856881 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:00.350564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9847a6-3655-4854-91ca-1c97c5856881 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:00.351802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 568/1107] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:00.356826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:00.357252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:00.357450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:00.376316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:00.377500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:00.441853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 540/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:00.716648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9847a6-3655-4854-91ca-1c97c5856881 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:00.814461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9847a6-3655-4854-91ca-1c97c5856881 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:00.846808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:00.847421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:00.847692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:00.847988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:00.849334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:00.850741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2223-e0e4-470f-a028-92480da4c4eb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:00.852309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 569/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:00:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:00.858360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:00.943693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:00.955012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a59598-4b25-4a96-b2f8-ca8ac9e0f4d5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:00.990855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:00.991043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:00.991317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:00.991586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:00.992253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:00.993679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387480e-9eab-4cba-bb44-a0fd09d410d9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:01.186330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 541/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 19:00:00 2022] POST /identity/v3/users => generated 286 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:03.103926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bcb2d-564b-454b-a3ea-abada6b79ca4 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:03.104467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bcb2d-564b-454b-a3ea-abada6b79ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:03.104467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bcb2d-564b-454b-a3ea-abada6b79ca4 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:03.105770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 570/1111] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:03.110953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:03.112006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:03.112208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:03.130416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:03.131648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:03.200465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 542/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:03.472760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bcb2d-564b-454b-a3ea-abada6b79ca4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:03.494939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:03.573137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bcb2d-564b-454b-a3ea-abada6b79ca4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:03.607668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:03.608323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:03.608603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:03.608876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] PATH_INFO: `/v3/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:03.610273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] RBAC: Authorizing `identity:get_role(role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:03.611637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:03.612985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 571/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:03 2022] GET /identity/v3/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:03.618857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:03.693120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:03.719309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db277c06-bbf0-4c8b-bbfa-035a1ff29041 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:03.755056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:03.755598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:03.756560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:03.756560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] PATH_INFO: `/v3/users/71015b165db04e9f9ceca167d06565b1` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:03.770910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] RBAC: Authorizing `identity:get_user(user_id=71015b165db04e9f9ceca167d06565b1)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:03.772393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:03.774049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 543/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:03 2022] GET /identity/v3/users/71015b165db04e9f9ceca167d06565b1 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:03.779837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:03.864788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:03.876135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d91149-c0e9-4ede-9d0f-e3bbc46d3c50 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:03.910114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:03.910710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:03.910982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:03.911246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:03.978224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:03.979746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:04.064674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:04.076487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 572/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:04.083161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:04.179938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ef002-bbf4-47cf-a169-bfa8cad760be None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:04.209219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:04.214086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:04.214710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.214986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.215255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.216468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:04.218787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7f747-6623-4048-9cdc-7c0a37aa2631 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:04.219581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 544/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:04 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:04.226335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:04.322488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d497ef-2628-4efa-ab7d-6e5e03796839 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:04.354803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:04.355394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.355668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.355964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.356677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009, user.id=71015b165db04e9f9ceca167d06565b1) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:04.356980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:04.358395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff0194-d8fb-4fe0-ae75-e33640fdc0f7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:04.364268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 573/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:04 2022] GET /identity/v3/role_assignments?user.id=71015b165db04e9f9ceca167d06565b1&scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:06.236118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63fdb-adb6-4cd0-b338-f42ad300a0a0 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:06.236661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63fdb-adb6-4cd0-b338-f42ad300a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:06.236661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63fdb-adb6-4cd0-b338-f42ad300a0a0 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:06.238012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 545/1118] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:06.243458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:06.243899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:06.243993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:06.262109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:06.263289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:06.311215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:06.331360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 574/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:06.596813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc63fdb-adb6-4cd0-b338-f42ad300a0a0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:06.695369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc63fdb-adb6-4cd0-b338-f42ad300a0a0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:06.728265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:06.728878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:06.729153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:06.729423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] PATH_INFO: `/v3/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:06.730785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] RBAC: Authorizing `identity:get_role(role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:06.732177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:06.733473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 546/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:06 2022] GET /identity/v3/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:00:06.740120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:06.835807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08309-2a22-4d4f-ac6d-b299a0c7e715 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:06.869656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:06.870210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:06.870485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:06.870757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] PATH_INFO: `/v3/users/71015b165db04e9f9ceca167d06565b1` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:06.871888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] RBAC: Authorizing `identity:get_user(user_id=71015b165db04e9f9ceca167d06565b1)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:06.873299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:06.874843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 575/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:06 2022] GET /identity/v3/users/71015b165db04e9f9ceca167d06565b1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:06.880751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:06.949181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:06.983352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e276e-5fd8-46e0-b3fa-2693a8d6e3f4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:07.021976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:07.022611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:07.022890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:07.023166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:07.024326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:07.025759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8829a-fcbc-4775-9dcf-6e93e551ad19 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:07.027395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 547/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:06 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:07.033360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:07.085778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:07.137276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef865d-1fa7-4845-861b-bdbc89d967bd None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:07.175082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:07.175733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:07.176073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:07.176324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009/users/71015b165db04e9f9ceca167d06565b1/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:07.178440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] RBAC: Authorizing `identity:create_grant(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009, user_id=71015b165db04e9f9ceca167d06565b1, role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:07.179895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38cab3-b741-4aa9-b95c-55b20a1a538e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:07.249432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 576/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 19:00:07 2022] PUT /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009/users/71015b165db04e9f9ceca167d06565b1/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:00:09.175582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1a576-e77c-4aff-ad5c-951f48ff0db2 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:09.176165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1a576-e77c-4aff-ad5c-951f48ff0db2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:09.176165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1a576-e77c-4aff-ad5c-951f48ff0db2 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:09.177481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 548/1124] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:09.182689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:09.183122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:09.183231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:09.202871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:09.204084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:09.274911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 577/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:09.544222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1a576-e77c-4aff-ad5c-951f48ff0db2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:09.661976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1a576-e77c-4aff-ad5c-951f48ff0db2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:09.696798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:09.697415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:09.697689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:09.697958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] PATH_INFO: `/v3/roles/234752f93e4d4cd1b228acd2ca7f982e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:09.699324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] RBAC: Authorizing `identity:get_role(role_id=234752f93e4d4cd1b228acd2ca7f982e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:09.700709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:09.702036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 549/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:09 2022] GET /identity/v3/roles/234752f93e4d4cd1b228acd2ca7f982e => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:09.707956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:09.748457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:09.803152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcee209-c10f-4b10-8bbe-b2e53ae9754d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:09.835756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:09.836386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:09.836856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:09.837335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] PATH_INFO: `/v3/users/71015b165db04e9f9ceca167d06565b1` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:09.839027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] RBAC: Authorizing `identity:get_user(user_id=71015b165db04e9f9ceca167d06565b1)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:09.840513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:09.842108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 578/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:09 2022] GET /identity/v3/users/71015b165db04e9f9ceca167d06565b1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:09.848422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:09.852436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:09.946130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19e1d0-30e2-4e78-86d7-9b4af37e928a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:09.979169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:09.979771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:09.980075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:09.980345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:10.045436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:10.046883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:10.050153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:10.141386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 550/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:10.147995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:10.244531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122e92c-d70f-4973-91c4-ce20b89d9f22 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:10.253329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:10.276815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:10.277428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:10.277701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:10.277966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] PATH_INFO: `/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:10.279049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] RBAC: Authorizing `identity:get_project(project_id=ea8238fa0b2c4384a1e5a7a99ee9d009)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:10.280492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab77f71-1c5d-4cb1-8f73-98f4e9d9837d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:10.282085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 579/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:10 2022] GET /identity/v3/projects/ea8238fa0b2c4384a1e5a7a99ee9d009 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:10.289077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:10.385570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afffdd-89e8-4c22-8ed8-10ed2a5ca858 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:10.387837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:10.417855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:10.418447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:10.418722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:10.418995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:10.419742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] RBAC: Adding query filter params (role.id=234752f93e4d4cd1b228acd2ca7f982e, scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009, user.id=71015b165db04e9f9ceca167d06565b1) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:10.420076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:10.421496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298c17-f2b4-4145-8f41-c4ad3ef02920 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:10.427610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 551/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:10 2022] GET /identity/v3/role_assignments?user.id=71015b165db04e9f9ceca167d06565b1&scope.project.id=ea8238fa0b2c4384a1e5a7a99ee9d009&role.id=234752f93e4d4cd1b228acd2ca7f982e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:12.336476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70985a3-a753-4e5b-85b4-b632ace2b9fe None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.337004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70985a3-a753-4e5b-85b4-b632ace2b9fe None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.337004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70985a3-a753-4e5b-85b4-b632ace2b9fe None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.338250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 580/1131] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:12.343359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.343738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.343924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.362871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:12.364058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:12.427173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 552/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:12.699466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70985a3-a753-4e5b-85b4-b632ace2b9fe None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:12.799245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70985a3-a753-4e5b-85b4-b632ace2b9fe None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:12.832779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:12.833393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.833672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.833942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.835037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:12.836490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b683a-a9b1-4c14-90a8-ffbcc97762ad None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:12.838024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 581/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:12.843832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:12.937434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:12.941412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8e521-61af-4800-b429-aad12e2cc78b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:12.975906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:12.976442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.976721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.976989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.977641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:12.979070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346282fd-6b27-4269-8b05-b083435d04ae None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:13.088030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 553/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 19:00:12 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:15.067317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167bef8-ae4e-4685-8723-878a86fc3bf4 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:15.067919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167bef8-ae4e-4685-8723-878a86fc3bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:15.067919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167bef8-ae4e-4685-8723-878a86fc3bf4 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:15.069201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 582/1135] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:15.074578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:15.075065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:15.075155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:15.094351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:15.095616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:15.169809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 554/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:15.440217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167bef8-ae4e-4685-8723-878a86fc3bf4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:15.489087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:15.550718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167bef8-ae4e-4685-8723-878a86fc3bf4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:15.586828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:15.587516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:15.587791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:15.588088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:15.589208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:15.590645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f28bc0-2fe7-4b3b-b4a1-b4f7171348a3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:15.592266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 583/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:00:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:15.598723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:15.683528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:15.703223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66daee-eb5f-4a85-9bef-79f9ab5538f5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:15.738593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:15.739200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:15.739474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:15.739744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:15.740424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:15.741866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32f3dc-e65b-4dbc-b20e-629deb1e362f None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:15.889297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 555/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 19:00:15 2022] POST /identity/v3/users => generated 286 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:17.860626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a955-e447-40a4-8018-9266f7f841f9 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:17.861158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a955-e447-40a4-8018-9266f7f841f9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:17.861158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a955-e447-40a4-8018-9266f7f841f9 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:17.863021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 584/1139] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:17.868561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:17.868976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:17.869102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:17.888209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:17.889414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:17.955945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 556/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:18.221874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a955-e447-40a4-8018-9266f7f841f9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:18.324634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a955-e447-40a4-8018-9266f7f841f9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:18.338671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:18.360975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:18.363092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:18.363092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:18.363092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:18.382228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:18.387418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:18.388643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:18.389568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 585/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:18.394812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:18.459553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:18.494489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74ab1b-6013-4ed9-a71f-3a586d4b3b06 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:18.528020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:18.528623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:18.528936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:18.529211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:18.529861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:18.530158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:18.531644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:18.540703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 557/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:18.547301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:18.651052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcb61-9fea-45a5-8e57-1de653d7246f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:18.685050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:18.685744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:18.686025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:18.686300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] PATH_INFO: `/v3/users/e33fa6cc71844a3697b30d3e6a03b838` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:18.700020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=e33fa6cc71844a3697b30d3e6a03b838)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:18.701458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:18.703093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 586/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:18 2022] GET /identity/v3/users/e33fa6cc71844a3697b30d3e6a03b838 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:18.709197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:18.812813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72b88-d73a-4bf4-8e73-24c5e28389e3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:18.840946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:18.848700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:18.849639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:18.849915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:18.850187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:18.917672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:18.919119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:19.023327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 558/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:19.029484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:19.060282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:19.131917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0271b9-273e-4345-8b68-77c340864ee2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:19.166266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:19.166924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:19.167226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:19.167500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] PATH_INFO: `/v3/projects/d002065c59b647f4939cea881f57d505` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:19.168718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] RBAC: Authorizing `identity:get_project(project_id=d002065c59b647f4939cea881f57d505)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:19.170276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6575828-79fd-4285-9fe2-674281805486 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:19.172005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 587/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:19 2022] GET /identity/v3/projects/d002065c59b647f4939cea881f57d505 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:19.178939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:19.206722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:19.279761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e07a-19b2-4b22-ad4b-420d185d9f66 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:19.315508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:19.316142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:19.316422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:19.316713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:19.317418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=d002065c59b647f4939cea881f57d505, user.id=e33fa6cc71844a3697b30d3e6a03b838) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:19.317727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:19.319177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae061b4f-5a20-438b-a7ef-884dca0023a9 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:19.324911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 559/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:19 2022] GET /identity/v3/role_assignments?user.id=e33fa6cc71844a3697b30d3e6a03b838&scope.project.id=d002065c59b647f4939cea881f57d505&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:21.206453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e2659-6ed8-4767-81b7-591d8608428a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:21.207016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e2659-6ed8-4767-81b7-591d8608428a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:21.207016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e2659-6ed8-4767-81b7-591d8608428a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:21.208397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 588/1147] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:21.213713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:21.214157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:21.214258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:21.232677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:21.233864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:21.302189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 560/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:21.573042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e2659-6ed8-4767-81b7-591d8608428a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:21.675797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:21.680525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e2659-6ed8-4767-81b7-591d8608428a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:21.715023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:21.715653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:21.715957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:21.716232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:21.727028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:21.731867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:21.732953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:21.733833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 589/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:21.739878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:21.764396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:21.841667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46760-8f49-43d4-bacd-0526158feb7f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:21.877131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:21.877737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:21.878013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:21.878290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:21.878947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:21.879252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:21.880723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:21.889211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 561/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:21.895141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:21.999340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e7ede-cbc3-420c-8187-eb6820d01cd3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:22.033909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:22.034547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:22.034828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:22.035101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] PATH_INFO: `/v3/users/e33fa6cc71844a3697b30d3e6a03b838` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:22.036264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] RBAC: Authorizing `identity:get_user(user_id=e33fa6cc71844a3697b30d3e6a03b838)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:22.037695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:22.039280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 590/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:21 2022] GET /identity/v3/users/e33fa6cc71844a3697b30d3e6a03b838 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:22.045100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:22.133960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:22.149799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95dce-48a2-4402-95a1-8754c8f0a316 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:22.183795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:22.184520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:22.184797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:22.185066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] PATH_INFO: `/v3/projects/d002065c59b647f4939cea881f57d505` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:22.186181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] RBAC: Authorizing `identity:get_project(project_id=d002065c59b647f4939cea881f57d505)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:22.187625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d49f2-d569-4437-8cc8-af69c52a7a73 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:22.189288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 562/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:22 2022] GET /identity/v3/projects/d002065c59b647f4939cea881f57d505 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:22.195312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:22.224658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:22.299491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f304af-e416-4f63-841b-13093cf307c8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:22.336789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:22.337461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:22.337741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:22.338013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] PATH_INFO: `/v3/projects/d002065c59b647f4939cea881f57d505/users/e33fa6cc71844a3697b30d3e6a03b838/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:22.339960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d002065c59b647f4939cea881f57d505, user_id=e33fa6cc71844a3697b30d3e6a03b838, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:22.341405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520722-9940-4deb-b482-0951c8a428a8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:22.425103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 591/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 19:00:22 2022] PUT /identity/v3/projects/d002065c59b647f4939cea881f57d505/users/e33fa6cc71844a3697b30d3e6a03b838/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:00:24.334873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4c2a-a4d7-4091-a036-b19196f28d28 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:24.335375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4c2a-a4d7-4091-a036-b19196f28d28 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:24.335375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4c2a-a4d7-4091-a036-b19196f28d28 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:24.336748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 563/1154] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:24.342124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:24.342623 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:24.342726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:24.362123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:24.363311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:24.430645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 592/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:24.699395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed4c2a-a4d7-4091-a036-b19196f28d28 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:24.800402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed4c2a-a4d7-4091-a036-b19196f28d28 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:24.836838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:24.837465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:24.837743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:24.838013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:24.848173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:24.853286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:24.855019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:00:24.856770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 564/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:24.862233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:24.884636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:24.967545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a92c7-0ad5-4b21-9e83-3e5f98d3be20 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:25.002744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:25.003358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.003637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.003937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.004635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:25.004937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:25.006309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:25.014456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 593/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:25.021072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:25.036279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:25.121891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2366b-a7b7-4506-af23-9a4eba6bbccb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:25.155206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:25.155833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.156131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.156398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] PATH_INFO: `/v3/users/e33fa6cc71844a3697b30d3e6a03b838` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.157616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] RBAC: Authorizing `identity:get_user(user_id=e33fa6cc71844a3697b30d3e6a03b838)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:25.159780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:25.161398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 565/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:25 2022] GET /identity/v3/users/e33fa6cc71844a3697b30d3e6a03b838 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:25.167637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:25.247162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:25.268350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b55ead-5f81-436e-b4c3-b57e4389fbf2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:25.302213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:25.302809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.303083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.303351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.372310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:25.373768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:25.451787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:25.473760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 594/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 19:00:25.480270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:25.559258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:25.580817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b264c-ef14-47e8-8850-7b7e71432a13 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:25.613948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:25.614560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.614832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.615099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] PATH_INFO: `/v3/projects/d002065c59b647f4939cea881f57d505` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.616216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] RBAC: Authorizing `identity:get_project(project_id=d002065c59b647f4939cea881f57d505)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:25.617645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7c4c5-cb80-4c59-bbcd-debda26ac63e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:25.619262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 566/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:25 2022] GET /identity/v3/projects/d002065c59b647f4939cea881f57d505 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:25.625556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:25.732791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe468-a695-4ed2-bd5d-d5e02803fad5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:25.765641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:25.766225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.766498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.766762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.768160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=d002065c59b647f4939cea881f57d505, user.id=e33fa6cc71844a3697b30d3e6a03b838) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:25.768461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:25.769865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b0fd-3242-497e-bab5-20bb61a41b5c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:25.775365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 595/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:25 2022] GET /identity/v3/role_assignments?user.id=e33fa6cc71844a3697b30d3e6a03b838&scope.project.id=d002065c59b647f4939cea881f57d505&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:27.680699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00e81-b467-42fc-a9b0-b4bb379888f8 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:27.681292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00e81-b467-42fc-a9b0-b4bb379888f8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:27.681292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00e81-b467-42fc-a9b0-b4bb379888f8 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:27.682554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 567/1162] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:27.688288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:27.688734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:27.688821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:27.707721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:27.708968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:27.747912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:27.778862 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 596/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:28.046784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00e81-b467-42fc-a9b0-b4bb379888f8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:28.149137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00e81-b467-42fc-a9b0-b4bb379888f8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:28.184253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:28.184856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:28.185133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:28.185405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:28.202682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:28.256237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:28.257618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:28.360427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 568/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:28.366460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:28.468707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6536-92f3-404b-97b9-4b50f5a0f938 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:28.507459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:28.508024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:28.508299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:28.508563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:28.515082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:28.516536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-d76ba159-5b8b-4dd2-ad0f-a45305833b86 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:28.528340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:28.529495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:28.530317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 597/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 19:00:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:28.535515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:28.584749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:28.635261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4887b3-2a60-4fff-9193-02612b8a3eaf None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:28.668750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:28.669347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:28.669625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:28.669897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:28.670529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:28.670832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:28.672239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5ff09-1d50-42d0-b4c8-644e86ffd0dc None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:28.679269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 569/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Aug 29 19:00:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:30.613571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e1e5-558e-4240-b3ec-b360969ba51b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:30.614172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e1e5-558e-4240-b3ec-b360969ba51b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:30.614172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e1e5-558e-4240-b3ec-b360969ba51b None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:30.615417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 598/1167] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:30.621096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9537-330a-4f12-b1be-3ff9711a67ad None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:30.621585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9537-330a-4f12-b1be-3ff9711a67ad None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:30.621692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9537-330a-4f12-b1be-3ff9711a67ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:30.645377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8e3c9537-330a-4f12-b1be-3ff9711a67ad None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:30.647002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8e3c9537-330a-4f12-b1be-3ff9711a67ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:30.723242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 570/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:31.004771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709e1e5-558e-4240-b3ec-b360969ba51b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:31.037968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:31.110717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709e1e5-558e-4240-b3ec-b360969ba51b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:31.146438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:31.147048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:31.147327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:31.147598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:31.148297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:31.149666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fdec6-e5ca-45b0-9bfa-95a85748946c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:31.249877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 599/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Mon Aug 29 19:00:31 2022] POST /identity/v3/domains => generated 267 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:33.194898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1f6-1f70-420e-97ad-e3842f54e49b None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:33.195586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1f6-1f70-420e-97ad-e3842f54e49b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:33.195586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1f6-1f70-420e-97ad-e3842f54e49b None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:33.196933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 571/1170] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:33.202614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:33.203065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:33.203155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:33.221402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:33.222590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:33.288989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 600/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:33.557340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e1f6-1f70-420e-97ad-e3842f54e49b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:33.603253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:33.661874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e1f6-1f70-420e-97ad-e3842f54e49b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:33.696015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:33.696637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:33.696910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:33.697176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] PATH_INFO: `/v3/domains/84256f4c156e4be6960ed490ce2fe634` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:33.708254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=84256f4c156e4be6960ed490ce2fe634)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:33.709701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246326c1-b6c7-4034-8a32-efb5e930a060 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:33.711348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 572/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Mon Aug 29 19:00:33 2022] GET /identity/v3/domains/84256f4c156e4be6960ed490ce2fe634 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:33.719596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:33.749036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:33.827695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f52da7-8327-4ba5-911e-5d6350a7cc22 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:33.861298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:33.861898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:33.862174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:33.862447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:33.863104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:33.864556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6d632-0648-4cf1-b35c-8f22a552df4d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:33.941152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 601/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 19:00:33 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:35.907868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee790-9fb8-4b43-b499-84d14c83bb54 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:35.908462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee790-9fb8-4b43-b499-84d14c83bb54 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:35.908462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee790-9fb8-4b43-b499-84d14c83bb54 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:35.909810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 573/1174] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:35.915144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:35.915598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:35.915700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:35.934461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:35.935645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:36.002711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 602/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:36.273545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bee790-9fb8-4b43-b499-84d14c83bb54 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:36.380976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bee790-9fb8-4b43-b499-84d14c83bb54 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:36.380976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:36.420793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:36.421432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:36.421709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:36.421982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] PATH_INFO: `/v3/domains/84256f4c156e4be6960ed490ce2fe634` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:36.423785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=84256f4c156e4be6960ed490ce2fe634)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:36.425262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00ac24-8f87-4edd-b131-d96b56c21207 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:36.426848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 574/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Mon Aug 29 19:00:36 2022] GET /identity/v3/domains/84256f4c156e4be6960ed490ce2fe634 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:36.432687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:36.435811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:36.531965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412f51e-4d58-461d-b14b-4359d91a53bc None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:36.565798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:36.566402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:36.566673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:36.566937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:36.567582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:36.569031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451e960-d3aa-4718-bfba-9fde3d9d5433 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:36.670063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 603/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 19:00:36 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:00:38.657925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9b83-7da8-412d-a726-024460d9907d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.658473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9b83-7da8-412d-a726-024460d9907d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.658473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9b83-7da8-412d-a726-024460d9907d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.659774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 575/1178] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:38.665335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.665766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.665859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.683776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:38.684997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:38.741146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:38.752774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 604/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:39.026186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9b83-7da8-412d-a726-024460d9907d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.129704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9b83-7da8-412d-a726-024460d9907d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.162492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.163114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.163390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.163662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.174231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:39.179038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:39.180253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:00:39.181314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 576/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:39.186294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.291110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914af30-bd01-4dca-9d2f-f6392164601d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.324395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.324995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.325269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.325541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.326195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:39.326496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:39.327892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:39.336181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 605/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:39.341824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.373113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.443927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a104a-f8f4-4a64-9779-4795e6adcac4 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.478906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.479541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.479820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.480122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] PATH_INFO: `/v3/users/744177bfa2e44601b0bee1c6ed33b255` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.494346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] RBAC: Authorizing `identity:get_user(user_id=744177bfa2e44601b0bee1c6ed33b255)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:39.495786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:39.497636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 577/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/users/744177bfa2e44601b0bee1c6ed33b255 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:39.504205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.558480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.606740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c552128-b7fc-4b19-8c2d-5886e0c502a8 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.639718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.640349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.640624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.640891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.711114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:39.712605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:39.765001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.815064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 606/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:39.821229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.908761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.920444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae14ec-990a-4569-ac48-62e1e66c9a4e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.954989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.955672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.955980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.956249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] PATH_INFO: `/v3/projects/2a19426dbc1d40679ee832565cb5f12d` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.957424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] RBAC: Authorizing `identity:get_project(project_id=2a19426dbc1d40679ee832565cb5f12d)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:39.958855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c904913-38aa-488f-b012-832f291cd839 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:39.960523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 578/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/projects/2a19426dbc1d40679ee832565cb5f12d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:39.966618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:40.067284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5557fa-8164-4a13-a9bf-2af11dfa5c6b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:40.100764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:40.101368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:40.101648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:40.101922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:40.102670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=2a19426dbc1d40679ee832565cb5f12d, user.id=744177bfa2e44601b0bee1c6ed33b255) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:40.102976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:40.104468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceff94-10fd-44e7-9a7c-8a1d205f68a0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:40.109717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 607/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/role_assignments?user.id=744177bfa2e44601b0bee1c6ed33b255&scope.project.id=2a19426dbc1d40679ee832565cb5f12d&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:41.997973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25659d8-48b4-44ee-81a7-caa9d7d732af None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:41.998385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25659d8-48b4-44ee-81a7-caa9d7d732af None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:41.998541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25659d8-48b4-44ee-81a7-caa9d7d732af None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:41.999935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 579/1186] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:42.005552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.005959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.006127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.024691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:42.025904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:42.037684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:42.098235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 608/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:42.367690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25659d8-48b4-44ee-81a7-caa9d7d732af None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:42.469231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25659d8-48b4-44ee-81a7-caa9d7d732af None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:42.503294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:42.503941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.504217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.504486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.514980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:42.519808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:42.520947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:00:42.521849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 580/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:42.527149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:42.630826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fc94f-114f-4cfe-8ada-737d847419fd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:42.664806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:42.665401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.665673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.665937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.666632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:42.666946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:42.668377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:42.676624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 609/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:42.682934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:42.700937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:42.780740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b661-ff51-4a4e-8b09-0c1e6730d905 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:42.815546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:42.816193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.816471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.816739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] PATH_INFO: `/v3/users/744177bfa2e44601b0bee1c6ed33b255` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.817869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] RBAC: Authorizing `identity:get_user(user_id=744177bfa2e44601b0bee1c6ed33b255)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:42.819309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:42.820925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 581/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:42 2022] GET /identity/v3/users/744177bfa2e44601b0bee1c6ed33b255 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:42.826799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:42.838521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:42.928169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33432-5de6-40ae-aa64-562f806ec991 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:42.966520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:42.967146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.967422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.967692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] PATH_INFO: `/v3/projects/2a19426dbc1d40679ee832565cb5f12d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.968827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] RBAC: Authorizing `identity:get_project(project_id=2a19426dbc1d40679ee832565cb5f12d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:42.970901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b7d6d-36c4-4855-8556-8158d391efcf None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:42.972555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 610/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:42 2022] GET /identity/v3/projects/2a19426dbc1d40679ee832565cb5f12d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:42.978811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:43.060134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:43.075807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc7196-7db3-475a-8a16-e817174314f1 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:43.109792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:43.110430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:43.110703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:43.110972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] PATH_INFO: `/v3/projects/2a19426dbc1d40679ee832565cb5f12d/users/744177bfa2e44601b0bee1c6ed33b255/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:43.112869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a19426dbc1d40679ee832565cb5f12d, user_id=744177bfa2e44601b0bee1c6ed33b255, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:43.114296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735938c1-9049-4ad8-842b-b1bf621ee119 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:43.212849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 582/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 19:00:42 2022] PUT /identity/v3/projects/2a19426dbc1d40679ee832565cb5f12d/users/744177bfa2e44601b0bee1c6ed33b255/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:00:45.110848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395d50-ba99-49c1-8c8a-9f666ebfe5e6 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.111359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395d50-ba99-49c1-8c8a-9f666ebfe5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.111359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395d50-ba99-49c1-8c8a-9f666ebfe5e6 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.112842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 611/1193] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:45.118070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.118454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.118603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.136867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:45.138032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:45.202145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 583/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:00:45.469262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca395d50-ba99-49c1-8c8a-9f666ebfe5e6 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:45.547253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:45.570651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca395d50-ba99-49c1-8c8a-9f666ebfe5e6 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:45.603758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:45.604406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.604682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.604950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.615940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:00:45.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:00:45.621887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:00:45.622953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 612/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:00:45.627978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:45.698497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:45.723295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63adbc-346f-4aef-9b83-74daccdf6eac None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:45.757674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:45.758271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.758547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.758817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.759466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:45.759763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:45.761322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:45.770594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 584/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:45.776550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:45.874473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1134ac-87ca-4792-a894-899b1723a78f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:45.907681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:45.908325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.908601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.908866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] PATH_INFO: `/v3/users/744177bfa2e44601b0bee1c6ed33b255` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.909974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] RBAC: Authorizing `identity:get_user(user_id=744177bfa2e44601b0bee1c6ed33b255)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:45.911389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:45.912993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 613/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Aug 29 19:00:45 2022] GET /identity/v3/users/744177bfa2e44601b0bee1c6ed33b255 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:45.922204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:46.020055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304d90-e622-41aa-b2cb-e796aee2da59 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:46.052398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:46.052989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:46.053260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:46.053544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:46.056817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:46.120323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:46.121767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:46.218386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 585/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:00:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:00:46.224746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:46.228540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:46.323335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400eac-2996-452e-97e4-5b20db80d128 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:46.356038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:46.356653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:46.356936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:46.357207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] PATH_INFO: `/v3/projects/2a19426dbc1d40679ee832565cb5f12d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:46.358304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] RBAC: Authorizing `identity:get_project(project_id=2a19426dbc1d40679ee832565cb5f12d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:46.359725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293572c0-c3e5-40ea-92f6-f9bd859e7826 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:46.361410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 614/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Aug 29 19:00:46 2022] GET /identity/v3/projects/2a19426dbc1d40679ee832565cb5f12d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:46.367528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:46.401372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:46.463529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76109-a6ea-4eee-9e9e-c209d0740c30 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:46.497061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:46.497649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:46.497924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:46.498191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:46.498905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=2a19426dbc1d40679ee832565cb5f12d, user.id=744177bfa2e44601b0bee1c6ed33b255) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:00:46.499211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:46.501291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79bc06-cc44-4be7-8d72-f044b65e201b None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:46.506980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 586/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:00:46 2022] GET /identity/v3/role_assignments?user.id=744177bfa2e44601b0bee1c6ed33b255&scope.project.id=2a19426dbc1d40679ee832565cb5f12d&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:00:56.351940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=64 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:56.504309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=50 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:02.750842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd724406-ad79-48cd-87b5-c6d82cefe094 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:02.751684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd724406-ad79-48cd-87b5-c6d82cefe094 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:02.751684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd724406-ad79-48cd-87b5-c6d82cefe094 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:02.752808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 615/1201] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:02.758083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:02.758561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:02.758661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:02.777607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:02.778805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:02.851757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 587/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:03.131181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd724406-ad79-48cd-87b5-c6d82cefe094 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:03.262079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd724406-ad79-48cd-87b5-c6d82cefe094 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:03.298749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:03.299383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.299660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.300029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.311309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:03.312795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:03.328720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:03.330881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:03.332882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:03.332882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:03.332882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:03.332882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 616/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:03.336138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:03.444036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441029f-805a-4538-81cc-e71bd1567a18 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:03.477216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:03.483238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:03.484002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.484332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.484602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.485329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:03.485818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:03.488857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0590f5-7e09-4b4c-b36e-ead94f5d18be None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:03.502262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 588/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:03.508424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:03.562823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:03.611253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a55b2-bc89-40e8-aec5-7be43ff90388 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:03.644643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:03.645251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.645531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.645799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.646552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:03.648016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7b14b-1f5c-4758-ad99-28985c237c8c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:03.828405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 617/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Aug 29 19:02:03 2022] POST /identity/v3/users => generated 251 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:05.802183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57343-4543-44fd-be70-f47bec2c7e06 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:05.802738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57343-4543-44fd-be70-f47bec2c7e06 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:05.802738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57343-4543-44fd-be70-f47bec2c7e06 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:05.804107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 589/1206] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:05.809463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:05.809901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:05.810016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:05.829174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:05.830367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:05.899426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 618/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 19:02:06.167199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e57343-4543-44fd-be70-f47bec2c7e06 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.275994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e57343-4543-44fd-be70-f47bec2c7e06 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.317923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.318601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.318876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.319140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.329513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:06.334338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:06.335384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:06.336267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 590/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:06.341805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.384314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.450386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366735e-7914-4315-aba3-3ab83195fcd7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.488377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.488987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.489267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.489536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.490199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:06.490499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.491886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:06.500584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 619/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:06.506693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.561940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.611495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea47e-04a6-484a-a54e-2e4497ed0978 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.645436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.646068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.646342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.646610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.656962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.658461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:06.673199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:06.674382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:06.675312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:06.675312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:06.675312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:06.676008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 591/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:06.680438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.780744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.783589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbe95e-f520-4479-8844-bf9900b41977 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.816211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.817010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.817148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.817322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.817960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:06.818251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.819605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:06.831433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 620/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:06.837494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.942061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35120df-2977-4114-9938-cb8850c34740 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.943594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.978532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.979165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.979445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.979713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.987251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.988766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:07.001491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:07.004099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:07.005262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 592/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:07.008737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.111629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af9c0-b55a-481f-8f8a-b3a1de39391a None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.146765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.147370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.147646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.147961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.148682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:07.148983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.150399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.168377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 621/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:07.175133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.275105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0efa6-c99d-478f-83f3-97f5997f2e54 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.309725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.310329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.311235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.311504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.350395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:07.389827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.391229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.505483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 593/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:02:07.511521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.532447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:07.613510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bc62-db45-4578-9cb2-49fdfcab3cfc None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.648534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.649092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.649409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.649637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.660448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.661959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:07.677091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:07.679515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:07.680694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:07.680694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:07.680694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:07.680694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 622/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:07.684311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.745979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:07.784095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fe82-f176-4734-94cf-55a8010f2c4a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.819257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.819890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.820171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.820443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.821083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:07.821446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.822933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.835633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 594/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:07.841539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.916631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:07.948998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7218923-d527-418d-b6ed-d68419bdcb96 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.983296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.983964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.984238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.984507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.991896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.993347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:08.005121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:08.006556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:08.007473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 623/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:08.012556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:08.121303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a289f-2501-4715-9356-af3157f4a4d7 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:08.145571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:08.157705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:08.158428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:08.158703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:08.158968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:08.159692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:08.160024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:08.161527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec9697-ec47-4d69-9977-5dfa21142ec6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:08.173799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 595/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:08.179788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:08.283157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db577e5-f442-4858-914d-33b1b55fe87c None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:08.311831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:08.317467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:08.318069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:08.318370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:08.318664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:08.319364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=7bd6962ec6f5425091a001e8ee251616) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:08.319665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:08.321124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d06f76-bcb8-4311-98f9-a713268ed2bb None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:08.326716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 624/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:02:08 2022] GET /identity/v3/role_assignments?user.id=7bd6962ec6f5425091a001e8ee251616&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:10.291589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ca7-fd7a-4dd4-bfe8-201ddfa6369f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.292132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ca7-fd7a-4dd4-bfe8-201ddfa6369f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.292132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ca7-fd7a-4dd4-bfe8-201ddfa6369f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.293470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 596/1220] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:10.298805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.299221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.299336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.317256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:10.318502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:10.384685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 625/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:10.659525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0ca7-fd7a-4dd4-bfe8-201ddfa6369f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.764190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0ca7-fd7a-4dd4-bfe8-201ddfa6369f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.799942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.801069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.801069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.801069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.812541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:10.817231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:10.818338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:10.819497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:10.823201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 597/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 19:02:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:10.824310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.924566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac9b6b-0290-458e-8766-714369c06a85 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.949766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:10.961019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.961628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.961907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.962181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.963042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] RBAC: Adding query filter params (name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:10.963350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.964844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.973149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 626/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 19:02:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:10.979224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.024379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.081435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442ae2-47f4-4072-9b0e-5c43b2030d12 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.115383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.116039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.116319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.116592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.127690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.129176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:11.143377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:11.144610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:11.145579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:11.145579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:11.145579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:11.145579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 598/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:11.150391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.253264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39ef4-97c0-47b6-81c2-ee9e5bf0626b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.286868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.287539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.287815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.288119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.288769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:11.289071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.290473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.302813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 627/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:11.308856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.402939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.410658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce22ab-66d7-4448-8774-6392a5539813 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.447788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.448528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.448863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.449218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.457506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.459272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:11.475233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:11.477280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:11.479633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 599/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:11.481230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.535294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.606691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876245a-24ba-49d9-9464-4c1d613b88d5 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.646944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.647557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.647829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.648128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.648848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:11.649146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.650723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.669816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 628/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:11.675897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.778106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed5d2-c955-4fb2-8818-f29acab371a2 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.818124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.818663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.818938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.819207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.831328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.832919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:11.848276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:11.850312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:11.852099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:11.852099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:11.852099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:11.852099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 600/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:11.855971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.957949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c82ff7-32af-4136-824d-966d69614a8d None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.960335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.993258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.993909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.994138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.996883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.996883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:11.996883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.996883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.009316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 629/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:12.015793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.025428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:12.116154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e986-5cb1-4ada-aa4c-768394ae2481 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.149278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.149903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.150181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.150451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.158264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.159795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:12.172412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:12.173871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:12.175208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 601/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:12.179692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.301856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca561-1bd8-4f9e-93c8-80d05057de93 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.336399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.337056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.337258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.337522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.338207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:12.338504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.339944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1b14-d3a7-4337-be38-deafb5df15a2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.352852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 630/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:12.359293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.430136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:12.469085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66d5b-34b6-42ca-9957-77e9d30bd4f3 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.502137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.502668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.503230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.503230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/7bd6962ec6f5425091a001e8ee251616/roles/e39a5105a6f24c84b30f7f324812ab70` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.519599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=7bd6962ec6f5425091a001e8ee251616, role_id=e39a5105a6f24c84b30f7f324812ab70)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.521084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b02fc1-bb55-4b46-9365-4b81c7e1b03c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.621961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 602/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 19:02:12 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/7bd6962ec6f5425091a001e8ee251616/roles/e39a5105a6f24c84b30f7f324812ab70 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:14.538983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a4b5-b889-44ab-8c7a-ae4bf44b82ee None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.539529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a4b5-b889-44ab-8c7a-ae4bf44b82ee None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.539636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a4b5-b889-44ab-8c7a-ae4bf44b82ee None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.541026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 631/1233] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:14.546923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.547309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.547467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.565774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:14.566963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:14.637734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 603/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:14.906002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8a4b5-b889-44ab-8c7a-ae4bf44b82ee None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.932011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.013600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8a4b5-b889-44ab-8c7a-ae4bf44b82ee None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.049789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.050423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.050700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.050971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.061832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:15.067324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:15.068425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:15.069341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 632/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Aug 29 19:02:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:15.075288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.124940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.178937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00746092-f5d3-488f-a7fb-302c7742b79f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.211235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.211881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.212200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.212471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.213121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] RBAC: Adding query filter params (name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:15.213420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.214820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.223053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 604/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:15.229038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.314000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.329759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c479-3300-4a35-8ab0-7cbc200c73f1 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.363567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.364240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.364523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.364796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.375236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.376783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:15.392148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:15.393313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:15.394558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:15.394558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:15.394558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:15.394558 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 633/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:15.399292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.504217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b583e-c379-4df1-9cf3-2811539d9dee None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.514491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.540342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.540342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.540342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.540926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.541398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:15.541901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.543302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.556208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 605/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:15.562508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.671947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a06a-c8d2-46f9-ac27-ac5af15438f3 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.708629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.709266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.709541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.709812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.712556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.718431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.719895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:15.730969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:15.732197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:15.733432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 634/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:15.737994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.839136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b55a87-c300-4fb4-91b0-2fec6514925e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.873293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.873998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.874277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.874549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.875266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:15.875568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.877016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.894786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 606/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:15.900684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.015879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f295fd-8750-4ef3-a0d7-3cc121e80c09 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.049732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.050355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.050635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.050953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.116020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.130764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.132411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.245317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 635/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:02:16.251441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.285644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.370415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa58dd1-f0b4-4c95-9180-34fb14695958 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.408060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.408690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.408983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.409237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.420272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.421837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:16.437723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:16.438971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:16.444495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:16.444495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:16.444495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:16.444495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 607/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:16.445325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.542224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.548507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5be16-ba28-42c4-9b7a-3dcb565bb4b0 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.587949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.588680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.588987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.589263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.589959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:16.590345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.591807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.605125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 636/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:16.611580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.704523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.727644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967e14-6276-4450-94f7-626ade9c4837 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.767988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.768623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.768928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.769197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.777501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.779661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:16.792297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:16.793450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:16.795333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 608/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:16.800388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.909220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2ad4b-6f72-42d7-ab74-ca9714b2725f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.944105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.944105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.944105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.944541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.945001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:16.945304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.946769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfda0d-210d-40c2-b253-aa2bfbe20b4c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.960535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 637/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:16.967706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:17.073478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6aee-23e6-47b3-a7bc-0f22e7015d2e None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:17.108376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:17.109006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.109287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.109558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.110303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] RBAC: Adding query filter params (role.id=e39a5105a6f24c84b30f7f324812ab70, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=7bd6962ec6f5425091a001e8ee251616) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:17.110613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:17.112087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5858c2-0efc-4cd6-93c3-7ff6518e62c6 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:17.117691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 609/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/role_assignments?user.id=7bd6962ec6f5425091a001e8ee251616&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=e39a5105a6f24c84b30f7f324812ab70 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:19.114271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8ef6-de6a-40f2-94d9-c43f2344e001 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.114873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8ef6-de6a-40f2-94d9-c43f2344e001 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.114873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8ef6-de6a-40f2-94d9-c43f2344e001 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.116201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 638/1247] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:19.122022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.122528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.122617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.129278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.142503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.144300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.212336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 610/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 19:02:19.487172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8ef6-de6a-40f2-94d9-c43f2344e001 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.506311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.590246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8ef6-de6a-40f2-94d9-c43f2344e001 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.626966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.627631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.627943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.628216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.638938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:19.644135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:19.645260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:19.646120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 639/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:19.651549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.754283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aaedb-69fb-45c6-ba98-dac186ccc455 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.784055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.788748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.789352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.789627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.789895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.790551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:19.790850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.792244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.801310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 611/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:19.807485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.892305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.915821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85f3e7-1112-4a21-b140-78727bdcdf48 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.949080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.949713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.949997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.950265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.961636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.963128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:19.978513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:19.984516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:19.985754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:19.985754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:19.985754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:19.985754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 640/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:19.986040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.086202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d5288-9e01-4ac8-a63f-b4f0b26b2f41 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.120129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.120733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.121013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.121292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.121926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:20.122228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.123607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.136009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 612/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:20.142346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.243720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0d02-049c-4280-ae3e-2bd4cbf20c94 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.269803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.277453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.278082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.278364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.279146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.286782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.288246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:20.300122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:20.308011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:20.309169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 641/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:20.309857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.364418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.407807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b40d4-c100-4f62-903b-a1cc913e43fd None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.441007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.441601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.441878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.442149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.442865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:20.443168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.444731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.461533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 613/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:20.467826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.570352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1f50-951a-49bd-a240-b5e764e05994 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.609986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.610606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.610884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.611155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.670481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.682935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.684520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.786595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 642/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:02:20.792872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.894957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac7e3-3e4c-442b-b333-9bc3d7f39fdb None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.910037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.931406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.932080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.932401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.932648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.943587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.945117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:20.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:20.966211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:20.972672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:20.972672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:20.972672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:20.972672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 614/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:20.972950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.058910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:21.071029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dcac-e7b7-466d-89f8-2af16edde197 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.104959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.105561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.105834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.106100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.106738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:21.107038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.108436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.119711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 643/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:21.126119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.228081 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d7d14-9783-44f7-a5fb-3403d5a5ca88 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.263685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.264349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.264626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.264982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.272085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.273540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:21.285400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:21.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:21.287643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 615/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:21.292803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.413943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eaa44-176f-4ec5-96a3-37e9e110ea38 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.449274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.450004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.450284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.450592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.451285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:21.451586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.453051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e3808-0323-4e6c-9c58-7379a080545d None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.465781 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 644/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:21.472611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.501245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:21.575541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688e5ff-1053-4613-a367-1f0b75147d24 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.611488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.612120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.612401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.612673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.613391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=7bd6962ec6f5425091a001e8ee251616) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:21.613749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.615189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fef6ee-55e3-424e-b266-465056adf51c None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.620948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 616/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:02:21 2022] GET /identity/v3/role_assignments?user.id=7bd6962ec6f5425091a001e8ee251616&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:23.571827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc47c-048e-4a90-8f92-fcab2258b3f2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.572511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc47c-048e-4a90-8f92-fcab2258b3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.572511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc47c-048e-4a90-8f92-fcab2258b3f2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.573755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 645/1261] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:23.580146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.580571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.580896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.601477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:23.602684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:23.673412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 617/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:23.956979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc47c-048e-4a90-8f92-fcab2258b3f2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.972554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.084875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc47c-048e-4a90-8f92-fcab2258b3f2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.120305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.120937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.121217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.121491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.132037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:24.136893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:24.139448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:24.141241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 646/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:24.145057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.163057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.246308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7a4f-712f-47f9-a2ce-653f4386441d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.280810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.284610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.292432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 618/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:24.298905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.369206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.408364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4e09-c868-4017-9922-7836a0054872 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.447070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.447693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.448035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.448310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.459283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.460756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:24.476825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:24.477999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:24.479117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:24.479117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:24.479117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:24.479117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 647/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:24.483911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.564587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.583698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff88c7-0c4f-458b-814e-4e110549e868 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.619342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.619980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.620253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.620595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.621160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:24.621476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.622839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.634571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 619/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:24.642045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.762386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881a60-5202-4772-bcb4-85148c20ef1e None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.783591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.798946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.799606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.799900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.800171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.808302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.810343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:24.822671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:24.824800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:24.826913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 648/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:24.830367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.932438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a0b45-afc5-4c8f-ae24-bb7fc46bc34a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.966786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.967393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.967670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.967968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.968696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:24.968999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.970448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.980941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.990729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 620/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:24.996632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:25.109453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255ce5a-613b-4c00-bd57-5954a567db9f None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:25.144520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:25.145147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.145426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.145695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.157050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:25.158534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:25.173769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:25.174836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:25.175936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:25.175936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:25.175936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:25.180279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 649/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:25.181342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:25.282956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb60ad-9914-4557-bd45-b77c107de6a5 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:25.317491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:25.318093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.318369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.318647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.319279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:25.319578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:25.320972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:25.332577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 621/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:25.342973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:25.381919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:25.453332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bef61-39b7-416b-86f2-50268bb711f7 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:25.487756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:25.488409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.488687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.488959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.495474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:25.496979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:25.508272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:25.509171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:25.510501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 650/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:25.515351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:25.577531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:25.623844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b4937-fe73-4736-b636-807a1f9056df None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:25.659179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:25.659714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.660023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.661116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.661116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:25.661309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:25.662710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbb5e3-1e54-4030-9dd4-01ded2df9494 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:25.676445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 622/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:25.682606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:25.778871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:25.790920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca86f82-f25d-4b13-a52f-0a27a410f836 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:25.828857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:25.829542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.829820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.830093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] PATH_INFO: `/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/7bd6962ec6f5425091a001e8ee251616/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.832125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] RBAC: Authorizing `identity:create_grant(project_id=69c75d1e1ab2498f95e156d48272226d, user_id=7bd6962ec6f5425091a001e8ee251616, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:25.833638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1065941-6fb0-4796-a6f2-fc1a0f233c44 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:26.130244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 651/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Aug 29 19:02:25 2022] PUT /identity/v3/projects/69c75d1e1ab2498f95e156d48272226d/users/7bd6962ec6f5425091a001e8ee251616/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:28.039417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2abff-556e-4c35-9e7d-b8efa6369c2b None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:28.040014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2abff-556e-4c35-9e7d-b8efa6369c2b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:28.040014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2abff-556e-4c35-9e7d-b8efa6369c2b None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:28.041319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 623/1274] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:28.047407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:28.047837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:28.047997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:28.066971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:28.068220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:28.141837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 652/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:28.428333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2abff-556e-4c35-9e7d-b8efa6369c2b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:28.530856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2abff-556e-4c35-9e7d-b8efa6369c2b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:28.549992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:28.567320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:28.567999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:28.568275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:28.568543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:28.581320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:02:28.586216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:28.587179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:28.588049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 624/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Aug 29 19:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:28.593841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:28.676355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:28.694566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d8e5-b7a9-40cf-86b2-a00f8a780839 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:28.727774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:28.728407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:28.728688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:28.728961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:28.729621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:28.729945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:28.731442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:28.740855 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 653/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Aug 29 19:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:28.746766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:28.853004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd5d68-9e13-4d00-b0f4-4e40c6a6f1a6 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:28.885962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:28.886588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:28.886858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:28.887121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:28.898364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:28.899890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:28.914684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:28.916133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:28.917395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:28.917395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:28.917395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:28.917395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 625/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:28.922586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.027501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ef532-1f5b-471c-a9e7-f39cbbc3f243 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.064055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:29.064748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.064928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.065199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.065846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:29.066147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:29.067738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:29.081464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 654/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:29.087463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.120241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.192468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d19456-aff7-4f4f-8b8a-b0a1cfbab480 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.227596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:29.228404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.228717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.228989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.236541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:29.238025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:02:29.250813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:29.251992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:29.253159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 626/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:29.258119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.262281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.361256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd497e4-df92-4d6f-b00c-476d838ee5da None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.396720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:29.397329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.397604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.397871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.398668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:29.398912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:29.400438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:29.418228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 655/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:29.424496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.531155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.541567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd3af-d7a1-4f77-bda8-41977e489d49 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.577777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:29.578391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.578668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.578940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.654396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:29.655896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:29.746760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.758085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 627/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 19:02:29.764643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.827924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.865822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538dab-bdff-4442-b55c-5642f16ef9ba None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.900787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:29.901406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.901725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.901995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.912040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:29.913553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:02:29.927628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:29.928858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:02:29.929989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:02:29.929989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:02:29.929989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:29.929989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 656/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:29.935505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:30.035743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67039a-56b1-4eb4-9891-2e95ba4064ce None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:30.071609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:30.072405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:30.072684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:30.072961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:30.073641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:30.073939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:30.076068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:30.088092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 628/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:30.096697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:30.206854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e73c9-68ba-4c26-aed4-f8041c015013 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:30.220990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:30.241038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:30.241655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:30.241929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:30.242236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:30.250467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:30.251948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:02:30.263891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:30.265018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: ERROR keystone.server.flask.application Aug 29 19:02:30.266137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 657/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Aug 29 19:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:30.271202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:30.325470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:30.375089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d4e19-4386-4f6e-8297-ef170ec54c06 None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:30.407008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:30.407595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:30.407886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:30.408157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:30.408828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:30.409122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:30.410539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03a7-cb1c-479c-8417-88c9cb037e3d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:30.423445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 629/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Aug 29 19:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:30.430465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:30.536920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9ddf-c2f8-4398-bbb5-06cafa64cbd2 None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:30.571640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:30.572270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:30.572551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:30.572823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:30.573543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] RBAC: Adding query filter params (role.id=ae077013d56245c7869e76158c81fd53, scope.project.id=69c75d1e1ab2498f95e156d48272226d, user.id=7bd6962ec6f5425091a001e8ee251616) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:30.573850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:30.575839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea389-41ac-4594-8042-fb2811926264 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:30.581656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 658/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Aug 29 19:02:30 2022] GET /identity/v3/role_assignments?user.id=7bd6962ec6f5425091a001e8ee251616&scope.project.id=69c75d1e1ab2498f95e156d48272226d&role.id=ae077013d56245c7869e76158c81fd53 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:32.561242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8d65-aa0e-4cbc-993d-7259aa80a97b None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.561900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8d65-aa0e-4cbc-993d-7259aa80a97b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.561900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8d65-aa0e-4cbc-993d-7259aa80a97b None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.563228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 630/1288] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:32.569116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.582487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.582487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.593648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:32.595007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:32.627810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:32.666283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 659/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:32.943480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc8d65-aa0e-4cbc-993d-7259aa80a97b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.055049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc8d65-aa0e-4cbc-993d-7259aa80a97b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.068809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:33.094437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.095405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.095674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.096004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.096599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.098074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:33.110145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:33.111543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:33.112767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 631/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 19:02:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:33.117526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.221601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c543ca-4f64-4ea7-82ac-8b9aa4abd150 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.258024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.258636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.258906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.259172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.259812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:33.260158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.261520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b953a-1e2c-481d-b6e4-37d87fb1bf5e None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.266941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 660/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Aug 29 19:02:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:33.272919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.383309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972472a-c3d0-4ad1-9a3c-f1ccafa3111a None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.424263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.424875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.425153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.425427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] PATH_INFO: `/v3/services` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.426060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:33.426385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.427962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32827c7a-1cc5-4bc0-b202-818fcbf6446a None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.433054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 632/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Aug 29 19:02:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:35.553333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c14961-af2a-4322-9891-710b9a0d96f2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.554045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c14961-af2a-4322-9891-710b9a0d96f2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.554045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c14961-af2a-4322-9891-710b9a0d96f2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.555438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 661/1293] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:35.561893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023df2-6b6f-4645-b2a7-65d46255e356 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.562676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023df2-6b6f-4645-b2a7-65d46255e356 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.562676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023df2-6b6f-4645-b2a7-65d46255e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.583578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-42023df2-6b6f-4645-b2a7-65d46255e356 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:35.584939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-42023df2-6b6f-4645-b2a7-65d46255e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.618827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.669150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 633/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:35.946818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c14961-af2a-4322-9891-710b9a0d96f2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.992148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:36.072761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c14961-af2a-4322-9891-710b9a0d96f2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.111440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.112088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.112367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.112638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.113298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.114686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca103e34-d463-47e0-82a9-98ca7d3b811c None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.190627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 662/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/services => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:38.158623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e25b11-c79c-4a87-b266-25307c83f1bc None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.159145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e25b11-c79c-4a87-b266-25307c83f1bc None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.159145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e25b11-c79c-4a87-b266-25307c83f1bc None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.160528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 634/1296] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:38.166924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.169714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.169714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.187017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:38.188300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.277963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 663/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:38.547073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e25b11-c79c-4a87-b266-25307c83f1bc None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.657440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e25b11-c79c-4a87-b266-25307c83f1bc None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.692523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.693159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.693439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.693738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.694333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.695846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:38.705849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:38.714878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:38.716517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 635/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:38.717349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.732377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.811361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba314f-5a47-4e52-b171-cb3c7c203ee9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.847152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.847770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.848078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.848345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.848979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:38.849529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.850910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.856726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 664/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:38.862960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.897077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.975165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da5cb-9cb8-4b2e-bac4-b2e2c21f2ebf None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.011201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.011805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.012114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.012385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.013077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] RBAC: Adding query filter params (interface=public, service_id=004e2e1b1218459d94e4677459e1b120, region_id=RegionOne) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:39.013374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.014739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77648f6e-b6de-4d34-88ba-cbc51d84cd06 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.020245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 636/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/endpoints?service_id=004e2e1b1218459d94e4677459e1b120&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:39.026025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:39.111880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:39.128738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa664-635f-4aab-9d53-58965b322d5b None admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.169294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.170182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.170182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.170396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.171000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.172395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e3085-6564-4823-a392-ddfb28788db0 None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.179271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 665/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Aug 29 19:02:39 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:41.122764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c01bd-0bce-4d1a-951f-4931681be43f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.123554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c01bd-0bce-4d1a-951f-4931681be43f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.123554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c01bd-0bce-4d1a-951f-4931681be43f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.124703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 637/1302] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:41.130356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.130853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.130957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.149784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:41.152565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.221045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 666/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 19:02:41.489865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c01bd-0bce-4d1a-951f-4931681be43f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.592028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.600073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c01bd-0bce-4d1a-951f-4931681be43f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.640002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.640632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.640910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.641178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.641756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.643131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:02:41.652174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:02:41.654141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application with Lock( Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 19:02:41.655236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: ERROR keystone.server.flask.application Aug 29 19:02:41.667108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 638/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:02:41.667196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.738385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.760625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542707f-a0e5-4b71-a1c9-d7bf66177c86 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.794654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.795260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.795536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.795804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] PATH_INFO: `/v3/services` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.796460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:41.796761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.798123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf45851-5603-48fd-9bcb-9e2ff01646ca None admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.805563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 667/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:41.811474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.917445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8f758-e22e-41d9-b8f8-5c1f4529da9d None admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.950742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.958567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.958567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.958567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.958567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.958567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1948bfa-77da-49c2-a76b-5043e71c5ba3 None admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:42.055691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 639/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:51.891295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=30 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:52.050606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:56.988728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5156-efe0-4005-bd91-e94abba7b3d6 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:56.989313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5156-efe0-4005-bd91-e94abba7b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:56.989313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5156-efe0-4005-bd91-e94abba7b3d6 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:56.997138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943ce8-48f1-403d-a04c-3f37bec837ec None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:56.997641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943ce8-48f1-403d-a04c-3f37bec837ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:56.997723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943ce8-48f1-403d-a04c-3f37bec837ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:57.002412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 668/1307] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:11:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:11:57.087576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-49943ce8-48f1-403d-a04c-3f37bec837ec None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:57.089052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-49943ce8-48f1-403d-a04c-3f37bec837ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:57.220398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 640/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:57.573590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:57.574127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:57.574614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:57.578134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 669/1309] 10.222.0.62 () {54 vars in 965 bytes} [Mon Aug 29 19:11:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:11:57.585567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc101-1a3d-404b-b14a-c0cae53051f9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:57.585855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc101-1a3d-404b-b14a-c0cae53051f9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:57.586127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc101-1a3d-404b-b14a-c0cae53051f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:57.672762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b3ccc101-1a3d-404b-b14a-c0cae53051f9 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:57.674012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b3ccc101-1a3d-404b-b14a-c0cae53051f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:57.696813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:57.779907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 641/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Mon Aug 29 19:11:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:57.790087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:57.792222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:57.940092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac3384-cf80-425f-a086-ad8ab63bf708 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:58.033315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:58.033921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:58.034128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:58.034397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:58.036163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:58.107499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:58.189305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:58.191181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda1471-2599-4095-9bb6-bba11c2d6aa0 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:58.214447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 670/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:12:00.554150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13723-05e2-4e33-a87b-4b015f4e0ac0 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:00.554776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13723-05e2-4e33-a87b-4b015f4e0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:00.554776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13723-05e2-4e33-a87b-4b015f4e0ac0 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:00.556288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 642/1312] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:00.561830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065313-c8d9-449f-814e-c5b7fce78f93 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:00.562637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065313-c8d9-449f-814e-c5b7fce78f93 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:00.562637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065313-c8d9-449f-814e-c5b7fce78f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:00.651943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2e065313-c8d9-449f-814e-c5b7fce78f93 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:00.653297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2e065313-c8d9-449f-814e-c5b7fce78f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:00.777328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 671/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:01.077844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:01.078314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:01.078595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:01.080058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 643/1314] 10.222.0.62 () {54 vars in 965 bytes} [Mon Aug 29 19:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:01.087419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656994-545f-41d9-ac7c-b05ed0ddaaa8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:01.087913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656994-545f-41d9-ac7c-b05ed0ddaaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:01.088466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656994-545f-41d9-ac7c-b05ed0ddaaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:01.105309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:01.182771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-aa656994-545f-41d9-ac7c-b05ed0ddaaa8 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:01.184429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-aa656994-545f-41d9-ac7c-b05ed0ddaaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:01.275717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 672/1315] 10.222.0.62 () {58 vars in 1062 bytes} [Mon Aug 29 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:01.285046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:01.286654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:01.402531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30a91a-51b1-4847-a677-b23a4c056609 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:01.483146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:01.486142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:01.486811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:01.487104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:01.487409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:01.489146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:01.612808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:01.614126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3da0c-b3ce-471f-a75a-72ca21b98af5 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:01.634822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 644/1316] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:04.575772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:04.576496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:04.576496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:04.577661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 673/1317] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:04.583119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b414-3abd-40f6-bfdb-9016b8ff0716 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:04.583568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b414-3abd-40f6-bfdb-9016b8ff0716 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:04.583678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b414-3abd-40f6-bfdb-9016b8ff0716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:04.653621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7a96b414-3abd-40f6-bfdb-9016b8ff0716 None None] MFA Rules not processed for user `9618dcb87a9040ebb47c80ebac85c6cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:04.654850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7a96b414-3abd-40f6-bfdb-9016b8ff0716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:04.710568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:04.756860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 645/1318] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:12:05.052164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:05.054425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:05.164928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1566-72a6-42f9-8554-3d2443b5af9a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:05.167771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:05.241720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:05.242222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:05.242493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:05.242751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:05.244506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:05.357419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:05.358896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3136e2-e618-46de-bbd6-0140c791dc00 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:05.378404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 674/1319] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:08.307640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:08.308199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:08.308199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:08.309468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 646/1320] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:08.314923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2801-d509-4e3b-a565-f23f61f4c52c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:08.315356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2801-d509-4e3b-a565-f23f61f4c52c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:08.315462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2801-d509-4e3b-a565-f23f61f4c52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:08.358049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:08.383930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ad3d2801-d509-4e3b-a565-f23f61f4c52c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:08.385175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ad3d2801-d509-4e3b-a565-f23f61f4c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:08.486847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 675/1321] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:08.784626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:08.786241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:08.893073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2271b2-24f7-4885-8a72-9273187454c2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:08.978857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:08.979378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:08.979650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:08.979949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:08.981800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:08.989578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:09.105850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:09.107563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7d946-4ebb-4087-958a-066636341fae service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:09.126395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 647/1322] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:12:12.541065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:12.541603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:12.541603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:12.542934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 676/1323] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:12.548457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8927-6ede-4b1a-a603-61d9031f5702 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:12.548866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8927-6ede-4b1a-a603-61d9031f5702 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:12.549001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8927-6ede-4b1a-a603-61d9031f5702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:12.622425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1fca8927-6ede-4b1a-a603-61d9031f5702 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:12.623655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1fca8927-6ede-4b1a-a603-61d9031f5702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:12.687264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:12.731700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 648/1324] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:13.035593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:13.037319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:13.186234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16141e-a2e2-46fd-9a89-9a7c571e33da None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:13.200938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:13.287144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:13.288380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:13.288666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:13.288940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:13.290630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:13.416526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:13.417996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b9c8-706e-46f0-99db-718aad124adb service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:13.436908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 677/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:15.506418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:15.506974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:15.506974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:15.508346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 649/1326] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:15.513903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5bca8-412e-4c93-bff9-6a85a9a8ea0f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:15.514334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5bca8-412e-4c93-bff9-6a85a9a8ea0f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:15.514437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5bca8-412e-4c93-bff9-6a85a9a8ea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:15.562576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:15.581277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-43f5bca8-412e-4c93-bff9-6a85a9a8ea0f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:15.582423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-43f5bca8-412e-4c93-bff9-6a85a9a8ea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:15.712131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 678/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:16.003415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:16.004479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:16.135772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b5da3-a27c-4e1c-ba81-c264abeb15a8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:16.236510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:16.240713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:16.240713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:16.240713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:16.240713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:16.259257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:16.389555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:16.391057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aad6d9-aff2-4023-a3be-d2f8eb6f65e2 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:16.414192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 650/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:18.454826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:18.455407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:18.455407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:18.456734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 679/1329] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:18.463082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d142bd-4ae0-4c42-9f83-0267427c52d9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:18.463592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d142bd-4ae0-4c42-9f83-0267427c52d9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:18.463687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d142bd-4ae0-4c42-9f83-0267427c52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:18.534646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-12d142bd-4ae0-4c42-9f83-0267427c52d9 None None] MFA Rules not processed for user `9618dcb87a9040ebb47c80ebac85c6cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:18.536019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-12d142bd-4ae0-4c42-9f83-0267427c52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:18.608252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:18.642709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 651/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:18.928339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:18.934841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:19.073830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6e388-b3b6-4882-a1e7-4ed6f6d23cc4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:19.095724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:19.159077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:19.159893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:19.160172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:19.160469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:19.162105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:19.289112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:19.290559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332f901-b197-44f8-a641-f971de99acc8 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:19.311973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 680/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:21.814490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:21.815070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:21.815070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:21.816719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 652/1332] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:21.824029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102f82-180b-426b-ad95-5af6b7ceaa4e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:21.824633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102f82-180b-426b-ad95-5af6b7ceaa4e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:21.824633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102f82-180b-426b-ad95-5af6b7ceaa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:21.895280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:21.899495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f0102f82-180b-426b-ad95-5af6b7ceaa4e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:21.900746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f0102f82-180b-426b-ad95-5af6b7ceaa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:22.000786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 681/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:22.294797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:22.296521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:22.404064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42495791-f682-4b36-a10c-0632c7ad3b5a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:22.481324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:22.482007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:22.482311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:22.482603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:22.484340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:22.513050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:22.594306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:22.595667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d29382-3d52-4ceb-97fc-9d0e772b1e3f service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:22.613976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 653/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:24.670372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:24.670970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:24.670970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:24.672286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 682/1335] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:24.677754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0bfc9-8b8e-4c6a-9f7a-b18196dfd1a2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:24.678190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0bfc9-8b8e-4c6a-9f7a-b18196dfd1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:24.678317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0bfc9-8b8e-4c6a-9f7a-b18196dfd1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:24.751211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-83b0bfc9-8b8e-4c6a-9f7a-b18196dfd1a2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:24.752458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-83b0bfc9-8b8e-4c6a-9f7a-b18196dfd1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:24.838855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:24.863040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 654/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:12:25.156547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:25.158153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:25.264284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009aaaa-f538-471e-a97d-71358aac110e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:25.293485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:25.342046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:25.342651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:25.342924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:25.343193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:25.344825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:25.458112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:25.459588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ffd0c-7f96-464b-b61c-187107432128 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:25.478629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 683/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:28.195181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:28.195734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:28.195734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:28.197159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 655/1338] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:28.202671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9e65-3ecb-4451-a77b-1980df27877e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:28.203095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9e65-3ecb-4451-a77b-1980df27877e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:28.203215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9e65-3ecb-4451-a77b-1980df27877e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:28.273742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-62eb9e65-3ecb-4451-a77b-1980df27877e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:28.274879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-62eb9e65-3ecb-4451-a77b-1980df27877e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:28.283251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:28.388506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 684/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:12:28.676314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:28.678430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:28.794351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c96a-2bc5-411a-9547-d1f8fbff7a97 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:28.874770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:28.875372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:28.875646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:28.875935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:28.877532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:28.919257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:29.002947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:29.004470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaf0fa-7368-441f-8600-3254553b96b8 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:29.024595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 656/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:33.624159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:33.624678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:33.624678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:33.626011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 685/1341] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:33.631469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5431-34f4-48f4-8d43-6456f6bc2724 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:33.631912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5431-34f4-48f4-8d43-6456f6bc2724 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:33.632057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5431-34f4-48f4-8d43-6456f6bc2724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:33.700871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4c9c5431-34f4-48f4-8d43-6456f6bc2724 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:33.702122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4c9c5431-34f4-48f4-8d43-6456f6bc2724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:33.792929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:33.804257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 657/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:34.088766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:34.095226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:34.214604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aef80f-922b-4954-8364-5f371cf4ec21 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:34.263286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:34.304212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:34.304824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:34.305106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:34.305377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:34.306990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:34.425408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:34.426852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac212a3-84d5-4ad9-8139-4b2e0751eedc service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:34.449473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 686/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:37.800648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:37.801384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:37.801384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:37.802563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 658/1344] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:37.808198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c43e6-f6bf-439b-88e4-7fb8038db271 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:37.808634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c43e6-f6bf-439b-88e4-7fb8038db271 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:37.808731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c43e6-f6bf-439b-88e4-7fb8038db271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:37.886898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a82c43e6-f6bf-439b-88e4-7fb8038db271 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:37.888187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a82c43e6-f6bf-439b-88e4-7fb8038db271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:37.910207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:38.000499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 687/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:38.309082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:38.310549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:38.437493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2e980-d87d-4789-93b7-b0667dad7f76 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:38.531357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:38.532302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:38.532585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:38.532853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:38.534480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:38.584418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:38.665419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:38.667089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ea03-5af6-420a-b45a-e420a2c2132c service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:38.699242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 659/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:43.891083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.891623 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.892392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.893996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 688/1347] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:43.901259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758987e-bc87-4ae4-a19e-84b6ef9e4da2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.901891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758987e-bc87-4ae4-a19e-84b6ef9e4da2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.901891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758987e-bc87-4ae4-a19e-84b6ef9e4da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:44.112576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2758987e-bc87-4ae4-a19e-84b6ef9e4da2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:44.112576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2758987e-bc87-4ae4-a19e-84b6ef9e4da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:44.152569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:44.159942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 660/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:44.475669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:44.477343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:44.625333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673a751-b3ab-42f1-bff1-1a07a96a76ea None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:44.676716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:44.706066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:44.706679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:44.706948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:44.707217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:44.709001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:44.868365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:44.869829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabb6c7-9e56-4002-b8c8-441c0fbb1d63 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:44.892297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 689/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:50.441937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679c6c-cee6-4100-8536-7256c521217c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:50.442502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679c6c-cee6-4100-8536-7256c521217c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:50.446399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679c6c-cee6-4100-8536-7256c521217c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:50.457715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 661/1350] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:12:50 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:12:50.467903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322053-4472-4ceb-8f8b-8683803a4896 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:50.470413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322053-4472-4ceb-8f8b-8683803a4896 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:50.470920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322053-4472-4ceb-8f8b-8683803a4896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:50.610970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d2322053-4472-4ceb-8f8b-8683803a4896 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:50.610970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d2322053-4472-4ceb-8f8b-8683803a4896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:50.647307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:50.793775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 690/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:12:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:12:56.106774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0402-d363-4198-a90b-e3ef18af5ecb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:56.107342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0402-d363-4198-a90b-e3ef18af5ecb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:56.107544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0402-d363-4198-a90b-e3ef18af5ecb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:56.109098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 662/1352] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:56.114496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d484-50b6-4f18-b024-5fc5b17a2686 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:56.114958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d484-50b6-4f18-b024-5fc5b17a2686 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:56.115063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d484-50b6-4f18-b024-5fc5b17a2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:56.221080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d2f9d484-50b6-4f18-b024-5fc5b17a2686 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:56.222226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d2f9d484-50b6-4f18-b024-5fc5b17a2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:56.316494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:56.364639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 691/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:57.711240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007740b0-8fa7-4599-973c-92164d2f9d52 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:57.712556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007740b0-8fa7-4599-973c-92164d2f9d52 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:57.713329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007740b0-8fa7-4599-973c-92164d2f9d52 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:57.715669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 663/1354] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:12:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:12:57.721675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267bfa1-a5fe-4ab9-a005-d29682bcdcff None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:57.722291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267bfa1-a5fe-4ab9-a005-d29682bcdcff None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:57.722291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267bfa1-a5fe-4ab9-a005-d29682bcdcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:57.820236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9267bfa1-a5fe-4ab9-a005-d29682bcdcff None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:57.821497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9267bfa1-a5fe-4ab9-a005-d29682bcdcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:57.964260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 692/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:12:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:13:01.864902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a89f2-66bc-40c2-a42c-abadc02cefbb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:01.865747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a89f2-66bc-40c2-a42c-abadc02cefbb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:01.865747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a89f2-66bc-40c2-a42c-abadc02cefbb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:01.867448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 664/1356] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:01.872991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464aa6-1798-4041-a7ff-d5f1a5eb67f8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:01.873693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464aa6-1798-4041-a7ff-d5f1a5eb67f8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:01.873693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464aa6-1798-4041-a7ff-d5f1a5eb67f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:01.912837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:01.950148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0c464aa6-1798-4041-a7ff-d5f1a5eb67f8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:01.951696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0c464aa6-1798-4041-a7ff-d5f1a5eb67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:02.097895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 693/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:02.465725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:02.466202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:02.466721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:02.468601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 665/1358] 10.222.0.62 () {54 vars in 962 bytes} [Mon Aug 29 19:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:02.478423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91c7c-7883-42e7-8e5a-8329fed460a5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:02.478423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91c7c-7883-42e7-8e5a-8329fed460a5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:02.478905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91c7c-7883-42e7-8e5a-8329fed460a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:02.570764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-3bd91c7c-7883-42e7-8e5a-8329fed460a5 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:02.572320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3bd91c7c-7883-42e7-8e5a-8329fed460a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:02.640533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:02.749710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 694/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Aug 29 19:13:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:02.757519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:02.759055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:02.925777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f18af7-b57c-4da9-a4db-ba3e05bc7b71 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:03.021624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:03.065750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:03.066391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:03.067069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:03.067869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:03.070164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:03.185065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:03.186690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a033-8115-4457-af53-198f9a6f1b27 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:03.209446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 666/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:13:05.731892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:05.732336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:05.732427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:05.733933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 695/1361] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:05.739079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503fe8-b3fb-4de1-a503-ff7b76edceb8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:05.739601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503fe8-b3fb-4de1-a503-ff7b76edceb8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:05.739601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503fe8-b3fb-4de1-a503-ff7b76edceb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:05.813720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fc503fe8-b3fb-4de1-a503-ff7b76edceb8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:05.814857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fc503fe8-b3fb-4de1-a503-ff7b76edceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.855618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:05.935922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 667/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:13:06.246565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:06.248679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:06.453661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdffe2-a3d5-4638-a0f4-fb864ba0a492 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:06.464183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:06.629745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:06.630800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:06.631307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:06.631737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:06.634870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:06.786043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:06.788676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb9190-bffc-4a08-985c-5b049e5c8467 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:06.811145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 696/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:09.319255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d2fb6-62dc-45c2-99fa-50a365a0d2ec None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:09.320284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d2fb6-62dc-45c2-99fa-50a365a0d2ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:09.320559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d2fb6-62dc-45c2-99fa-50a365a0d2ec None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:09.321949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 668/1364] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:09.327968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2583-c25a-41c8-b92a-1c0a94ce9873 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:09.328483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2583-c25a-41c8-b92a-1c0a94ce9873 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:09.328483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2583-c25a-41c8-b92a-1c0a94ce9873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:09.346959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:09.414087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7a5d2583-c25a-41c8-b92a-1c0a94ce9873 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:09.415300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7a5d2583-c25a-41c8-b92a-1c0a94ce9873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:09.547706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 697/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:09.864345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:09.864991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:09.865480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:09.867384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 669/1366] 10.222.0.62 () {54 vars in 962 bytes} [Mon Aug 29 19:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:09.872944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929878d-7947-46da-ad2a-b10fb95fc339 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:09.873291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929878d-7947-46da-ad2a-b10fb95fc339 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:09.880204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929878d-7947-46da-ad2a-b10fb95fc339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:09.977162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2929878d-7947-46da-ad2a-b10fb95fc339 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:09.978854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2929878d-7947-46da-ad2a-b10fb95fc339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:10.027430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:10.163302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 698/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Aug 29 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:10.174506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:10.178060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:10.383332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28c25b-822b-41f0-ab86-c4ccea9be4af None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:10.415588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:10.532816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:10.533439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:10.533707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:10.533983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:10.536213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:10.647742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:10.649791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af0167-1984-43f5-974c-b456b35f1c33 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:10.683210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 670/1368] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:11.163299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d60751-4ea5-4eff-b97b-1b0b1b31f4b5 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.164173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d60751-4ea5-4eff-b97b-1b0b1b31f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.164173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d60751-4ea5-4eff-b97b-1b0b1b31f4b5 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.165608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 699/1369] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:11.171237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e42b-4956-4a8d-98ba-44ae70294637 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.171551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e42b-4956-4a8d-98ba-44ae70294637 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.171902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e42b-4956-4a8d-98ba-44ae70294637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.210541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:11.252961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7032e42b-4956-4a8d-98ba-44ae70294637 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:11.255079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7032e42b-4956-4a8d-98ba-44ae70294637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:11.374693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 671/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:11.980391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.980391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.980391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.980391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 700/1371] 10.222.0.62 () {54 vars in 964 bytes} [Mon Aug 29 19:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:11.983215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ffebb-8394-49b9-aa49-68d67e6ed6ec None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.983215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ffebb-8394-49b9-aa49-68d67e6ed6ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.983215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ffebb-8394-49b9-aa49-68d67e6ed6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:12.071531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4c5ffebb-8394-49b9-aa49-68d67e6ed6ec None None] MFA Rules not processed for user `bb59243400b44b8e998daaa4bd8f1ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:12.072786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4c5ffebb-8394-49b9-aa49-68d67e6ed6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:12.143773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:12.179211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 672/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:12.189680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:12.191994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:12.316467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81ac7a-ba5a-42c4-9e7d-7845b142f229 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:12.360695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:12.401137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:12.401903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:12.402119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:12.403141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:12.404357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:12.532113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:12.533963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ef7eb-f25a-4699-aebd-bd89b73c88e5 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:12.561072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 701/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:13:13.082284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:13.082819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:13.082903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:13.084966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 673/1374] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:13:13.090494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc98b91-b3c8-4c41-9aea-63749288b7b7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:13.090830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc98b91-b3c8-4c41-9aea-63749288b7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:13.091108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc98b91-b3c8-4c41-9aea-63749288b7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:13.164530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2dc98b91-b3c8-4c41-9aea-63749288b7b7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:13.165791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2dc98b91-b3c8-4c41-9aea-63749288b7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:13.187772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:13.276330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 702/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:13.571119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:13.573258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:13.758680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c964c-94c7-4ce0-bacf-3948a287d092 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:13.814107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:13.879514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:13.880154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:13.880434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:13.880707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:13.882369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:13.973791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:13.975601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d469c-a09c-4612-a9ae-18fbbabf7b6b service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:13.995023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 674/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:16.421006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:16.421526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:16.421526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:16.423009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 703/1377] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:16.429170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ef5d-5e5a-4d69-9af8-0a59c9648d14 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:16.429656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ef5d-5e5a-4d69-9af8-0a59c9648d14 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:16.429879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ef5d-5e5a-4d69-9af8-0a59c9648d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:16.499643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3869ef5d-5e5a-4d69-9af8-0a59c9648d14 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:16.500878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3869ef5d-5e5a-4d69-9af8-0a59c9648d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:16.508602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:16.617050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 675/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:16.929527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:16.931036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:17.100421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07044c92-00fb-4bc3-82fb-8878f8e158ae None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:17.104895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:17.235401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:17.236510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:17.237165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:17.237784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:17.239895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:17.342018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:17.343569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ca569-7839-4dc1-ace6-abf800ed47db service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:17.364837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 704/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:19.693796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:19.694536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:19.694868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:19.700069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 676/1380] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:19.706450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707fda-fa30-40cc-9c8b-7ff55f9c2357 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:19.707280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707fda-fa30-40cc-9c8b-7ff55f9c2357 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:19.707427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707fda-fa30-40cc-9c8b-7ff55f9c2357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:19.726359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:19.821826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9d707fda-fa30-40cc-9c8b-7ff55f9c2357 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:19.823708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9d707fda-fa30-40cc-9c8b-7ff55f9c2357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:19.968935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 705/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:20.266155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:20.268024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:20.435939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:20.447966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90194fce-3963-4f4a-9f11-e004b128b6a0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:20.574147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:20.575134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:20.575628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:20.576120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:20.577942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:20.676104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:20.678013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbbbe-2f52-45c1-b67e-31ae2cb038da service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:20.688386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:20.698267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 677/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:23.369116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:23.369855 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:23.369855 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:23.371039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 706/1383] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:23.378492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3adbe1-ae72-436d-8a81-73f1a2242432 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:23.379203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3adbe1-ae72-436d-8a81-73f1a2242432 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:23.379700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3adbe1-ae72-436d-8a81-73f1a2242432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:23.469401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3b3adbe1-ae72-436d-8a81-73f1a2242432 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:23.470673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3b3adbe1-ae72-436d-8a81-73f1a2242432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:23.591937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 678/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:23.887238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:23.888909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:24.010741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:24.088052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee601fff-28a5-45c5-8cce-b75a49e979c2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:24.244430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:24.245258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:24.245621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:24.245979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:24.248167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:24.312489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:24.359391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:24.361229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6669a2b-d3d0-445e-a678-e3fb8f4da1df service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:24.383043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 707/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:26.958694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:26.959265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:26.959265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:26.960955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 679/1386] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:26.966535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012c1f-7f53-4136-96ae-073fee01ffcc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:26.967063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012c1f-7f53-4136-96ae-073fee01ffcc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:26.967063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012c1f-7f53-4136-96ae-073fee01ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:27.045270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b7012c1f-7f53-4136-96ae-073fee01ffcc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:27.046514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b7012c1f-7f53-4136-96ae-073fee01ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:27.153142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 708/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:27.443740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:27.445274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:27.528069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:27.604839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c7169-4679-405a-a74e-8c66d2191c55 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:27.751106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:27.751716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:27.752028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:27.752323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:27.754489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:27.793141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:27.848761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:27.850367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed80db-4e1f-4fd8-984e-211f151d6898 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:27.870865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 680/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:28.956739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea3e0-ada7-4bc5-b4a7-38e962211fde None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:28.957575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea3e0-ada7-4bc5-b4a7-38e962211fde None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:28.957575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea3e0-ada7-4bc5-b4a7-38e962211fde None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:28.958800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 709/1389] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:28.964828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9475ed-349d-4bd8-ade6-4901df5f265a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:28.965367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9475ed-349d-4bd8-ade6-4901df5f265a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:28.965469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9475ed-349d-4bd8-ade6-4901df5f265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:29.048085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ec9475ed-349d-4bd8-ade6-4901df5f265a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:29.049333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ec9475ed-349d-4bd8-ade6-4901df5f265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:29.166705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 681/1390] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:29.482319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:29.484358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:29.484358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:29.484358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 710/1391] 10.222.0.62 () {54 vars in 967 bytes} [Mon Aug 29 19:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:29.490511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11c4-6866-4013-934d-44f3b6eb7874 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:29.490790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11c4-6866-4013-934d-44f3b6eb7874 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:29.491061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11c4-6866-4013-934d-44f3b6eb7874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:29.509344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:29.589855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8ddb11c4-6866-4013-934d-44f3b6eb7874 None None] MFA Rules not processed for user `fa271ae95240497e9be3175e44a2b545`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:29.591115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8ddb11c4-6866-4013-934d-44f3b6eb7874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:29.682873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 682/1392] 10.222.0.62 () {58 vars in 1064 bytes} [Mon Aug 29 19:13:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:13:29.691305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:29.692857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:29.728416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:29.813761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3c396-ccf5-41a8-8a53-460b3675b024 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:29.897583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:29.898194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:29.898470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:29.898737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:29.900373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:29.988093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:30.020582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:30.022098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e076-a3f1-4fa4-a675-aedf342b4651 service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:30.041426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 711/1393] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:30.141575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015046-3305-4a61-af49-2f1df888307d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:30.142133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015046-3305-4a61-af49-2f1df888307d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:30.142133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015046-3305-4a61-af49-2f1df888307d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:30.143995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 683/1394] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:13:30.149643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba72ec-e320-44e2-a422-2f70f6e931c6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:30.149920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba72ec-e320-44e2-a422-2f70f6e931c6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:30.150189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba72ec-e320-44e2-a422-2f70f6e931c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:30.222709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d8ba72ec-e320-44e2-a422-2f70f6e931c6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:30.224088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d8ba72ec-e320-44e2-a422-2f70f6e931c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:30.333905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 712/1395] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:30.628999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e015046-3305-4a61-af49-2f1df888307d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:30.631086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3e015046-3305-4a61-af49-2f1df888307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:30.741544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:30.794810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e015046-3305-4a61-af49-2f1df888307d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:30.919575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:30.925269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:30.925269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:30.925269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:30.925269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:31.001626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:31.030886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:31.032562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee58f2d-6101-48bf-98d2-6a0a85cdcb9a service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:31.053673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 684/1396] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:31.056482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7646-b604-4cf3-84d7-0b28c0fd38e6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:31.056964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7646-b604-4cf3-84d7-0b28c0fd38e6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:31.057114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7646-b604-4cf3-84d7-0b28c0fd38e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:31.105231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:31.129373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-20fb7646-b604-4cf3-84d7-0b28c0fd38e6 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:31.130657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-20fb7646-b604-4cf3-84d7-0b28c0fd38e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:31.222408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 713/1397] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:31.272918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:31.273474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:31.273590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:31.275150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 685/1398] 10.222.0.62 () {54 vars in 943 bytes} [Mon Aug 29 19:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:31.282341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1bda6-8512-4114-a739-09a5adad3f35 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:31.282633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1bda6-8512-4114-a739-09a5adad3f35 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:31.282908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1bda6-8512-4114-a739-09a5adad3f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:31.379636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d6d1bda6-8512-4114-a739-09a5adad3f35 None None] MFA Rules not processed for user `d045b7cdd6024b6b82f3c4864c01e7d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:31.380934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d6d1bda6-8512-4114-a739-09a5adad3f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:31.451713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:31.502677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 714/1399] 10.222.0.62 () {58 vars in 1040 bytes} [Mon Aug 29 19:13:31 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:31.512511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:31.514054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:31.707671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370896c9-3eeb-4ba6-a4a9-3cf817af910d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:31.800555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:31.862918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:31.863676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:31.864113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:31.864412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:31.866079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:31.965650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:31.967319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d6f6f-3f66-4ac5-b642-1b556ba414c1 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:31.980020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 686/1400] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:13:33.297801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:33.298331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:33.298331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:33.300796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 715/1401] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:33.307933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52d08-3399-4249-aa38-b8a374a1f287 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:33.307933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52d08-3399-4249-aa38-b8a374a1f287 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:33.307933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52d08-3399-4249-aa38-b8a374a1f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:33.389665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-63c52d08-3399-4249-aa38-b8a374a1f287 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:33.390910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-63c52d08-3399-4249-aa38-b8a374a1f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:33.442565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:33.514702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 687/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:33.810213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:33.812118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:33.977449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477fc2a-ac26-47f1-a924-eaa62a740917 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:34.040829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:34.118825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:34.119446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:34.119721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:34.120020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:34.121666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:34.234350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:34.235898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ac7f1-04bb-4885-be63-b8ecc5685dec service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:34.255888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 716/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:36.937834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:36.938365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:36.938526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:36.940747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 688/1404] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:36.947037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1a5d7-f311-4f04-ad24-f17df5100b60 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:36.947516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1a5d7-f311-4f04-ad24-f17df5100b60 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:36.947614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1a5d7-f311-4f04-ad24-f17df5100b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:37.022322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:37.035579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b8b1a5d7-f311-4f04-ad24-f17df5100b60 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:37.036905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b8b1a5d7-f311-4f04-ad24-f17df5100b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:37.178682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 717/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:37.474165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:37.475739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:37.653553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dac8f-b491-477e-ba15-8fbfcea1b786 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:37.684492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:37.791389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:37.792294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:37.792760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:37.793027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:37.794721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:37.900183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:37.901800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f15aa-f3ab-48c1-b43c-39f52fd6cc8b service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:37.923540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 689/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:40.291991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:40.292509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:40.292509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:40.293773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 718/1407] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:40.299734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4a7ea-428f-4f0e-86be-f0b8a1b4c728 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:40.300970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4a7ea-428f-4f0e-86be-f0b8a1b4c728 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:40.301460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4a7ea-428f-4f0e-86be-f0b8a1b4c728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:40.361233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:40.404912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9fb4a7ea-428f-4f0e-86be-f0b8a1b4c728 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:40.406290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9fb4a7ea-428f-4f0e-86be-f0b8a1b4c728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:40.561079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 690/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:40.907229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:40.911202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:41.086078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:41.118894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4dcac-e057-4f82-964d-8a86f8808583 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:41.299076 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:41.300323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:41.300323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:41.300323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:41.302130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:41.402506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:41.409075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:41.410553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f146d-0782-446c-b324-bad7b24ad56e service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:41.432912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 719/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:50.557855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=52 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:51.472324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:56.894436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374bc81-eaa7-43c5-84e0-d66ca488cf57 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:56.895014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374bc81-eaa7-43c5-84e0-d66ca488cf57 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:56.895124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374bc81-eaa7-43c5-84e0-d66ca488cf57 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:56.896514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 691/1410] 10.222.0.62 () {54 vars in 918 bytes} [Mon Aug 29 19:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:56.901730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202fda-c66d-4770-856e-f0e9da2cbd86 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:56.902322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202fda-c66d-4770-856e-f0e9da2cbd86 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:56.902322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202fda-c66d-4770-856e-f0e9da2cbd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:56.983421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-14202fda-c66d-4770-856e-f0e9da2cbd86 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:56.984791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-14202fda-c66d-4770-856e-f0e9da2cbd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:57.093140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 720/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Mon Aug 29 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:57.108383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e989d-9f13-4681-a21e-10d832b4a3ac None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:57.108662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e989d-9f13-4681-a21e-10d832b4a3ac None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:57.108933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e989d-9f13-4681-a21e-10d832b4a3ac None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:57.110264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 692/1412] 10.222.0.62 () {54 vars in 918 bytes} [Mon Aug 29 19:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:57.116585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09089a8-5e57-4385-8d02-3525fbc6d94f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:57.118513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09089a8-5e57-4385-8d02-3525fbc6d94f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:57.119146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09089a8-5e57-4385-8d02-3525fbc6d94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:57.205947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:57.209042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f09089a8-5e57-4385-8d02-3525fbc6d94f None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:57.210234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f09089a8-5e57-4385-8d02-3525fbc6d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:57.335169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 721/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Mon Aug 29 19:13:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:13:59.796794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:59.797400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:59.797400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:59.799182 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:13:59.808817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d967-8cb9-45b3-bbe8-3fa912b1defb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:59.809341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d967-8cb9-45b3-bbe8-3fa912b1defb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:59.809456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d967-8cb9-45b3-bbe8-3fa912b1defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:59.880756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9225d967-8cb9-45b3-bbe8-3fa912b1defb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:59.882011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9225d967-8cb9-45b3-bbe8-3fa912b1defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:59.943594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:59.987527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:00.283994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:00.285538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:00.436627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e352a89-d581-43c4-b44f-f3e8904f552a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:00.511722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:00.560835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:00.561350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:00.561624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:00.561896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:00.563509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:00.660088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:00.661712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7473bf-cfd7-4d09-9d1e-81e70e690801 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:00.681722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 694/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:01.604049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.604632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.604632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.605931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:01.612147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d957db-60f5-4dcb-909d-8782b707b32b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.612929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d957db-60f5-4dcb-909d-8782b707b32b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.612929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d957db-60f5-4dcb-909d-8782b707b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.696903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b4d957db-60f5-4dcb-909d-8782b707b32b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:01.698213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b4d957db-60f5-4dcb-909d-8782b707b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:01.728197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:01.805546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:01.960089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.961654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:02.138544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31e3bd-76c8-48eb-9562-bfacd6aed04b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.182875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:02.262949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.263479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.263761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.264083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.265714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:02.361815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.363480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca568f-aeae-4d26-a595-9c486718bf1d service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.382719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 724/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:03.276419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.277083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.277083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.278318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:14:03.283778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f70-4764-4085-bc5b-65539815a389 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.284246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f70-4764-4085-bc5b-65539815a389 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.284363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f70-4764-4085-bc5b-65539815a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.340615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:03.365057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-389d5f70-4764-4085-bc5b-65539815a389 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:03.366294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-389d5f70-4764-4085-bc5b-65539815a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.478149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:03.639950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.641497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.794922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ef050-7e3c-4b4a-a6aa-d194a64c64c9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.808193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:03.917700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.918368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.918643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.919018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.920622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:04.019051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.020731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef1fac-7239-4f93-8f1d-1191682d182c service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.040989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 697/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:04.934467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.935031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.935031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.936519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:04.941429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6350-4d5f-4cc9-9daf-a7defc3ddcd2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.941857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6350-4d5f-4cc9-9daf-a7defc3ddcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.941970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6350-4d5f-4cc9-9daf-a7defc3ddcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.962397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.029407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-12dc6350-4d5f-4cc9-9daf-a7defc3ddcd2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:05.031171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-12dc6350-4d5f-4cc9-9daf-a7defc3ddcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.151543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:05.302864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.304530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.431971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.471029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00bef7-299f-47f3-a4f1-8d143bc33d9c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.594113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.595084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.595493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.595902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.597758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.691776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.705981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.707595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d022e-8484-493b-8821-ac8f169c0032 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.727447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 727/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:06.618159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:06.618807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:06.618807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:06.620109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:06.625773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b30b-c887-499b-a5bf-7f9991281a1a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:06.626307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b30b-c887-499b-a5bf-7f9991281a1a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:06.626407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b30b-c887-499b-a5bf-7f9991281a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:06.713401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c8a3b30b-c887-499b-a5bf-7f9991281a1a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:06.714643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c8a3b30b-c887-499b-a5bf-7f9991281a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:06.819570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:06.972761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:06.974285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:07.063043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:07.132329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea248bd-aa5a-477e-80a6-51bcd3b4de57 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:07.248261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:07.248613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:07.248883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:07.249167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:07.250814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:07.304872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:07.347674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:07.349366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e9357-ffad-4625-86a2-0f907ef0278b service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:07.368236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 700/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:08.322029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:08.322576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:08.322576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:08.324166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:08.331807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03117746-d947-401b-b1e3-3a46090931f6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:08.332386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03117746-d947-401b-b1e3-3a46090931f6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:08.332781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03117746-d947-401b-b1e3-3a46090931f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:08.416799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-03117746-d947-401b-b1e3-3a46090931f6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:08.418047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-03117746-d947-401b-b1e3-3a46090931f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:08.521297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:08.676064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:08.677707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:08.732357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:08.838945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0f455-3e28-4f23-ae7b-e68b33191c53 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:08.957576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:08.958087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:08.958358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:08.958624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:08.960249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:08.967270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:09.064066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:09.065730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e22204-a420-47fd-8b74-c8052ab0099d service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:09.086047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 730/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:10.061781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.062313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.062313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.064308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:10.069506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd3f5-5f9c-4ef5-81d7-f893e495efbd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.069960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd3f5-5f9c-4ef5-81d7-f893e495efbd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.070071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd3f5-5f9c-4ef5-81d7-f893e495efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.153206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7a8cd3f5-5f9c-4ef5-81d7-f893e495efbd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:10.154418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7a8cd3f5-5f9c-4ef5-81d7-f893e495efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.222284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:10.253522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:10.417024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:10.418539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.436358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:10.563499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fdfec-1604-441d-9572-ee9922e453eb None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:10.663399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:10.690973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:10.691586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.691905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.692158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.693918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.908012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:10.909606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155b7e9-4c47-4cb4-9aa0-cb88e338222e service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:10.928400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 703/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:11.848662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.849235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.849235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.850530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:11.855617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f322172-d6d5-4f0a-aade-771163ccfd16 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.856105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f322172-d6d5-4f0a-aade-771163ccfd16 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.856202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f322172-d6d5-4f0a-aade-771163ccfd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.939317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8f322172-d6d5-4f0a-aade-771163ccfd16 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:11.940563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8f322172-d6d5-4f0a-aade-771163ccfd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:11.987148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:12.042780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:12.195634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.197186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.348612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ecb7-fb24-4e75-9812-6d2480b6da4b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.403398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:12.464550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.465152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.465428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.465699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.467299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.558344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:12.560013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6eeda-cb5e-4cb9-8f10-ec6269492df6 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.577532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 733/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:13.454577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.455136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.455136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.456521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:14:13.461555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd530206-e9d7-4af7-909d-2143c9fc3686 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.461971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd530206-e9d7-4af7-909d-2143c9fc3686 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.462101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd530206-e9d7-4af7-909d-2143c9fc3686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.538439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bd530206-e9d7-4af7-909d-2143c9fc3686 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:13.539651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bd530206-e9d7-4af7-909d-2143c9fc3686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.541741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:13.640501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:13.792587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.794024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.938736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4526a0-9416-41e0-975f-d198afb2e796 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.967310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:14.057338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:14.057882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:14.059019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:14.059019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:14.060054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:14.149742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:14.151384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ebc7c-425f-4ffd-a790-4ac8d7a04f2d service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:14.171779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 706/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:16.005439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.008972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.008972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.008972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:16.012764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e850e7-ebc1-4b44-954b-52b6bccd5d9f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.013226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e850e7-ebc1-4b44-954b-52b6bccd5d9f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.013332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e850e7-ebc1-4b44-954b-52b6bccd5d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.056962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:16.088105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-32e850e7-ebc1-4b44-954b-52b6bccd5d9f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:16.089339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-32e850e7-ebc1-4b44-954b-52b6bccd5d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.189908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:16.474330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.475878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.581414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa5fa6-c8d3-496d-afa9-20811a1bd0d7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.614571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:16.656998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.657620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.657904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.658663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.660308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.772932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.774296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89a84c-a059-4bd3-bbfd-33620949009b service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.794286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 736/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:18.900376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.900943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.900943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.902237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:18.907749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aced29-9f6f-4141-b0e4-045de9856d65 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.908261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aced29-9f6f-4141-b0e4-045de9856d65 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.908425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aced29-9f6f-4141-b0e4-045de9856d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.975233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a7aced29-9f6f-4141-b0e4-045de9856d65 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:18.976463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a7aced29-9f6f-4141-b0e4-045de9856d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:18.989858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:19.073611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:19.361730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.363251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:19.468430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417bb49-700c-4e04-8f98-53199f2b949f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.480644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:19.545462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.546064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.546341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.546609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.548241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:19.659346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.660727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de88dc-9550-480a-8ee7-6c0c6686227d service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.678958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 709/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:22.040951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.041476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.041476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.042810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:22.048361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056aa74-4507-47d7-b29c-df3a03d9af23 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.049390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056aa74-4507-47d7-b29c-df3a03d9af23 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.049548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056aa74-4507-47d7-b29c-df3a03d9af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.098855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.117895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3056aa74-4507-47d7-b29c-df3a03d9af23 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:22.119271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3056aa74-4507-47d7-b29c-df3a03d9af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.217150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:22.508573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.510123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.613271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad137-8d9d-481b-a68f-52abb46566f3 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.652449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.686268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.686895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.687199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.687417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.693057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.795218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.796600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2696cf2-51a4-4afb-b671-b31263a02e95 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.815405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 739/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:24.890736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.891304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.891304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.892659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:24.897975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2eca-9798-4518-9051-aadc0dec6c89 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.898386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2eca-9798-4518-9051-aadc0dec6c89 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.898539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2eca-9798-4518-9051-aadc0dec6c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.971037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e84c2eca-9798-4518-9051-aadc0dec6c89 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:24.972281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e84c2eca-9798-4518-9051-aadc0dec6c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:24.994339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:25.191997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:25.479672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.481275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:25.583305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d3e21-f73b-4246-9827-3e41ab2c6aa1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.602816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:25.658826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.659445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.659719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.660017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.661609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:25.769459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.770805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d49fa5-f61f-440e-8261-fae15459c5de service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.792714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 712/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:27.785089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:27.785613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:27.785613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:27.786923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:27.792143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349141-007d-42a2-a052-e0e0d4dffba1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:27.792560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349141-007d-42a2-a052-e0e0d4dffba1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:27.792730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349141-007d-42a2-a052-e0e0d4dffba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:27.859143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:27.862284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-95349141-007d-42a2-a052-e0e0d4dffba1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:27.863552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-95349141-007d-42a2-a052-e0e0d4dffba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:27.959418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:28.255258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:28.256841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:28.361106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e40a84-124d-48ac-be00-4537c09c3b74 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:28.412267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:28.435947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:28.437377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.437377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.437377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.438625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:28.550825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:28.552215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1f59d-203c-41da-bc72-3b72791fb2f1 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:28.570879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 742/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:30.573598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.574213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.574213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.575490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:30.581576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554081-4d78-43d8-ab2a-29fe0e56acd7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.582055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554081-4d78-43d8-ab2a-29fe0e56acd7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.582165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554081-4d78-43d8-ab2a-29fe0e56acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.648903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d8554081-4d78-43d8-ab2a-29fe0e56acd7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:30.650100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d8554081-4d78-43d8-ab2a-29fe0e56acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.679896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:30.746265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:31.036868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.038470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.150321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff116607-81c1-4cf5-ade2-eea3b275acf3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:31.178591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:31.224553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:31.225159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.225433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.225699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.227290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.337162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:31.338547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a6b3-dcbf-4f99-b2e7-c95e79dd6fbd service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.357784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 715/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:33.340787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.341362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.341362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.342660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:33.348544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a1f7-359d-4764-a400-6e0ef7289fbc None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.348971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a1f7-359d-4764-a400-6e0ef7289fbc None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.349178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a1f7-359d-4764-a400-6e0ef7289fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.417283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3592a1f7-359d-4764-a400-6e0ef7289fbc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:33.418587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3592a1f7-359d-4764-a400-6e0ef7289fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:33.427202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:33.517990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:33.803120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:33.804809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:33.909467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857fb9-ab55-4a97-b2b0-fdf536d418c1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:33.970802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:33.992784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:33.992784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.992784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.992784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.994319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:34.104908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:34.106242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf2ddc-7338-4c91-ba85-ac9b5b30bfd1 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:34.125824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 745/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:36.114977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.115557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.115557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.116948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:36.122771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87cf6-a6fb-40bf-b724-0640c89a2344 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.123233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87cf6-a6fb-40bf-b724-0640c89a2344 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.123345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87cf6-a6fb-40bf-b724-0640c89a2344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.196277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-def87cf6-a6fb-40bf-b724-0640c89a2344 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:36.197508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-def87cf6-a6fb-40bf-b724-0640c89a2344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.241626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:36.304405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:36.592061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:36.593767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.699619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e6e0d-c58a-445a-aee6-3033a97943de None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:36.738042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:36.773501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:36.774108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.774383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.774651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.776272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.886478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:36.887898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be23668-d431-4ad9-9c48-485f5983245a service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:36.905960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 718/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:38.889425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.889950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.889950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.891299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:38.897124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc0a82-7961-4648-b8d8-4a076e8b67d3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.897605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc0a82-7961-4648-b8d8-4a076e8b67d3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.897701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc0a82-7961-4648-b8d8-4a076e8b67d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.963201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cfcc0a82-7961-4648-b8d8-4a076e8b67d3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:38.964452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cfcc0a82-7961-4648-b8d8-4a076e8b67d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:38.983224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:39.064605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:39.351980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:39.353547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:39.462245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15b2e8-ea66-4efa-82e0-64dd4b690206 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:39.530773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:39.538005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:39.538619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:39.538890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:39.539155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:39.540814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:39.654292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:39.655653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164e265-2f1e-4a2e-8427-2d52f42ed5e4 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:39.674979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 748/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:41.631079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:41.631647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:41.631647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:41.633018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:41.638195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb849-d8e0-4f3b-8f92-07dd7701093b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:41.638601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb849-d8e0-4f3b-8f92-07dd7701093b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:41.638742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb849-d8e0-4f3b-8f92-07dd7701093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:41.706923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b10bb849-d8e0-4f3b-8f92-07dd7701093b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:41.708168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b10bb849-d8e0-4f3b-8f92-07dd7701093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:41.761304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:41.827534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:42.111194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:42.112753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:42.218617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a964b3-cfad-4fbd-a0fb-69bf7502ea93 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:42.270194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:42.300884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:42.301489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.301766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.302038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.303697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:42.417658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:42.419104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe1523-158f-457f-acaf-c2c18ca044f8 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:42.437233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 721/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:51.825722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=22 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:52.435035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=66 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:02.599024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dd30-b416-4621-afcc-3e7732df89e1 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.599607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dd30-b416-4621-afcc-3e7732df89e1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.599728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dd30-b416-4621-afcc-3e7732df89e1 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.601094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 750/1471] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:15:02.606610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd550-991f-4c2e-9dd1-0b1b4664dd09 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.607331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd550-991f-4c2e-9dd1-0b1b4664dd09 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.607424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd550-991f-4c2e-9dd1-0b1b4664dd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.680111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-34bbd550-991f-4c2e-9dd1-0b1b4664dd09 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:02.681345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-34bbd550-991f-4c2e-9dd1-0b1b4664dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:02.799844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 722/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:03.109104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:03.109382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:03.109648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:03.110984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 751/1473] 10.222.0.62 () {54 vars in 967 bytes} [Mon Aug 29 19:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:15:03.116075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb113-a21c-49d3-b7b3-dd345951a299 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:03.116359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb113-a21c-49d3-b7b3-dd345951a299 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:03.116667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb113-a21c-49d3-b7b3-dd345951a299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:03.194732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-395bb113-a21c-49d3-b7b3-dd345951a299 None None] MFA Rules not processed for user `fa271ae95240497e9be3175e44a2b545`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:03.195989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-395bb113-a21c-49d3-b7b3-dd345951a299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:03.216617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:03.273742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 723/1474] 10.222.0.62 () {58 vars in 1064 bytes} [Mon Aug 29 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:03.281698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:03.283252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:03.389276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8642c4-1a54-46f4-899b-8d87d25416d4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:03.462047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:03.462652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:03.462984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:03.463213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:03.464979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:03.525024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:03.584849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:03.591090 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dcffb-7349-4196-8047-3d5d993283f3 service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:03.607626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 752/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:05.691759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:05.692376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:05.692376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:05.693798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 724/1476] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:15:05.699531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee466db8-45fe-40e4-bbac-961e3ef68793 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:05.700107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee466db8-45fe-40e4-bbac-961e3ef68793 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:05.700107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee466db8-45fe-40e4-bbac-961e3ef68793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:05.774180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ee466db8-45fe-40e4-bbac-961e3ef68793 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:05.775403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ee466db8-45fe-40e4-bbac-961e3ef68793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:05.886156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 753/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:06.177493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:06.179040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:06.316948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ac35d-ba69-40d7-8bac-fb8e03946e59 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:06.371818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:06.423764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:06.424486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:06.424769 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:06.425046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:06.425704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:06.427539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:15:06.427539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:15:06.427539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:15:06.428718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815ba2-3ca6-4af4-8142-891b92bfc917 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:06.454013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 725/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Aug 29 19:15:06 2022] GET /identity/v3/projects => generated 2553 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:08.453557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:08.454154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:08.454154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:08.455477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 754/1479] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:15:08.461426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26965fb4-b9bf-45d9-89fc-5730ca98a222 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:08.461924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26965fb4-b9bf-45d9-89fc-5730ca98a222 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:08.462019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26965fb4-b9bf-45d9-89fc-5730ca98a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:08.538168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-26965fb4-b9bf-45d9-89fc-5730ca98a222 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:08.539516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-26965fb4-b9bf-45d9-89fc-5730ca98a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:08.652381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 726/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:08.951817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:08.953385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:08.964322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:09.113084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf121f-3656-42ac-b967-34b84b2a597a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:09.206465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:09.245350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:09.246303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.246687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.247296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.249235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:09.361369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:09.363808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b57b4-86ef-42ef-9f5d-5dd4d99af257 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:09.388901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 755/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:11.606362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:11.607162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:11.607162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:11.608719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 727/1482] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:15:11.614539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b77-d71b-44b7-9dc4-aa1e86941c18 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:11.615020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b77-d71b-44b7-9dc4-aa1e86941c18 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:11.615118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b77-d71b-44b7-9dc4-aa1e86941c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:11.702263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-73314b77-d71b-44b7-9dc4-aa1e86941c18 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:11.704474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-73314b77-d71b-44b7-9dc4-aa1e86941c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:11.758419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:11.819400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 756/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:12.108365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:12.116763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:12.143235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:12.236995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236270f4-921d-4c86-9c79-372690a9a0d0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:12.328468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:12.329002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:12.329283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:12.329586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:12.331293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:12.418161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:12.458803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:12.460283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081de2-696c-4b19-b3be-a9bf176a1750 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:12.480213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 728/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:14.610261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:14.610829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:14.610829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:14.612192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 757/1485] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:15:14.623156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390a9f-ed16-4142-b73f-1be04738d9c5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:14.623768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390a9f-ed16-4142-b73f-1be04738d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:14.623768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390a9f-ed16-4142-b73f-1be04738d9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:14.709631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ec390a9f-ed16-4142-b73f-1be04738d9c5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:14.711019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ec390a9f-ed16-4142-b73f-1be04738d9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:14.819964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 729/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:15.118573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:15.120143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:15.242716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16d3e7-20b5-4194-9f60-8e6ecfe5d47b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:15.339523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:15.341570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:15.342176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:15.342447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:15.342714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:15.344414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:15.469972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:15.471486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50fec-4f67-497f-a761-38893341e472 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:15.491215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 758/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:17.733302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:17.734018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:17.734018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:17.735259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 730/1488] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:15:17.740828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1837bc-49d4-471a-bd0b-6fa16320fef1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:17.741351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1837bc-49d4-471a-bd0b-6fa16320fef1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:17.741351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1837bc-49d4-471a-bd0b-6fa16320fef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:17.816572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-da1837bc-49d4-471a-bd0b-6fa16320fef1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:17.817808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-da1837bc-49d4-471a-bd0b-6fa16320fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:17.873054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:17.916918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 759/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:18.214502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:18.216766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:18.257544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:18.325046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dabdf-b6c6-437b-9aca-dc15abb58395 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:18.397868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:18.398466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:18.398739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:18.399002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:18.400618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:18.485447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:18.515385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:18.516776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d9b7-fd1e-4e00-a418-cd7a99d631b6 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:18.534871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 731/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:21.059475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.060147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.060147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.061587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 760/1491] 10.222.0.62 () {54 vars in 927 bytes} [Mon Aug 29 19:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:15:21.067916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae38e8-9021-477e-b07d-530590eb0426 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.068732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae38e8-9021-477e-b07d-530590eb0426 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.068732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae38e8-9021-477e-b07d-530590eb0426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.148393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-57ae38e8-9021-477e-b07d-530590eb0426 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:21.149559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-57ae38e8-9021-477e-b07d-530590eb0426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.267214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 732/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:21.561184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.563312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.668940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf93845-8c56-43ec-bfe5-a90e74047f54 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.745459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.746067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.746336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.746604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.748227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.757888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:21.860785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.862399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47e93-5417-4cfe-8469-e7460d7eeae8 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.881783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 761/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:31.264809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=82 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:31.879465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=50 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:54.947378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:54.948026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:54.948026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:55.040177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:55.041439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:55.158667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 733/1494] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:55.167221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:55.168345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:55.168345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:55.252302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:55.253555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:55.368611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 762/1495] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:55.373348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:55.374559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:55.386792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfec1b1-7eb1-46ea-b559-bee29898a9c6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:55.457967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:55.496103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:55.496722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:55.496996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:55.497269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:55.497916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:55.498215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:55.499246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:55.499246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:16:55.499246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:16:55.500298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:55.513693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 734/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:55.522948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:55.539470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198ccf2-7367-443d-8c4a-ae22618c4fa4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:55.608042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:55.644215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:55.644821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:55.645095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:55.645359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:55.645994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:55.647736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:55.647736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:16:55.647736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:16:55.648958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:55.857581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 763/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:55 2022] POST /identity/v3/projects => generated 377 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:55.862187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:55.863876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:55.904159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1664b-c01c-4fd2-ba26-3bf9590115d1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:56.002096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:56.003011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.003011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.003305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.003840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:56.004892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:56.004892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:16:56.004892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:16:56.006122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:56.242729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 735/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:55 2022] POST /identity/v3/users => generated 289 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:56.247456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:56.249105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:56.316342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832551e0-74b8-46d6-88c2-eee872236d0c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:56.403683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:56.415200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:56.415799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.416100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.416423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.417064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:56.418604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:56.418604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:16:56.418604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:16:56.419483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:56.432352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 764/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:56.437645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:56.440230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:56.456947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b5bdd-d07e-4fc9-a6e0-4f3922845dce admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:56.483970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:56.555804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:56.556555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.556864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.557125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.557877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:56.559517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:56.559517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:16:56.559517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:16:56.560771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ccb33-b6ee-4c71-8487-2c1a3322f0f2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:56.575381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 736/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:56.579874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:56.581467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:56.617306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7321b8-879c-4786-adea-fccb9ffc0185 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:56.735747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:56.736281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.736568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.736840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] PATH_INFO: `/v3/projects/8a0f7a23b2854ea68ddf01c09a056d24/users/246e179eeb5642b0bee9f5f078ff35f6/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.763712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f7a23b2854ea68ddf01c09a056d24, user_id=246e179eeb5642b0bee9f5f078ff35f6, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:56.764790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:56.764790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:16:56.764790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:16:56.766761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af7a1-55c3-4ec8-9d2c-5ac00cac36da admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:56.916430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 765/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:56 2022] PUT /identity/v3/projects/8a0f7a23b2854ea68ddf01c09a056d24/users/246e179eeb5642b0bee9f5f078ff35f6/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:56.929730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0038-76b4-4a4f-bd45-4ebe708ae3a4 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.930113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0038-76b4-4a4f-bd45-4ebe708ae3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.930401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0038-76b4-4a4f-bd45-4ebe708ae3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.992638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-aaea0038-76b4-4a4f-bd45-4ebe708ae3a4 None None] MFA Rules not processed for user `246e179eeb5642b0bee9f5f078ff35f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:56.993877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-aaea0038-76b4-4a4f-bd45-4ebe708ae3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.103357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 737/1502] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:57.117823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.118114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.118405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.119778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 766/1503] 10.222.0.62 () {54 vars in 964 bytes} [Mon Aug 29 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:16:57.126767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102df1d-7829-4eb6-9423-e5347fd1aef7 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.127087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102df1d-7829-4eb6-9423-e5347fd1aef7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.127356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102df1d-7829-4eb6-9423-e5347fd1aef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.167985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:57.231987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6102df1d-7829-4eb6-9423-e5347fd1aef7 None None] MFA Rules not processed for user `bb59243400b44b8e998daaa4bd8f1ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:57.233459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6102df1d-7829-4eb6-9423-e5347fd1aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.324824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 738/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Mon Aug 29 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:57.333016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:57.334533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.400343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:57.464974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274ceb-3df6-47ce-ab4f-04c831af27d3 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:57.544178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:57.544804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.545075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.545349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.546985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.670730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:57.672432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:57.676108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:57.696974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 767/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:57.749238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca554049-0759-4e22-8521-ce35937e1c7f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.749554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca554049-0759-4e22-8521-ce35937e1c7f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.750050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca554049-0759-4e22-8521-ce35937e1c7f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.752648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 739/1506] 10.222.0.62 () {52 vars in 830 bytes} [Mon Aug 29 19:16:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:16:57.756648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:57.758598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.874864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c877466-e794-48e2-a734-803c578901cc service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:57.983632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:57.984202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.984473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.984739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] PATH_INFO: `/v3/users/246e179eeb5642b0bee9f5f078ff35f6` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.999670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246e179eeb5642b0bee9f5f078ff35f6)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:58.000760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:58.000760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:16:58.000760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:16:58.001992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:58.030674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:58.348497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246e179eeb5642b0bee9f5f078ff35f6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:58.349170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246e179eeb5642b0bee9f5f078ff35f6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:58.376677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246e179eeb5642b0bee9f5f078ff35f6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:58.393057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246e179eeb5642b0bee9f5f078ff35f6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:58.502303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246e179eeb5642b0bee9f5f078ff35f6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:58.509887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 768/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:57 2022] DELETE /identity/v3/users/246e179eeb5642b0bee9f5f078ff35f6 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:58.514944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:58.515495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:58.515602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:58.562273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:58.563585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:58.628673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 740/1508] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:58.639977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:58.641610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:58.740519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fe64-ea61-40f0-bfaf-77fc5fa956a1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:58.744575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:58.745220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:58.745601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:58.745878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:58.755533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:58.757015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a795a-0c3d-4dd3-a478-1d88f382ac01 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:58.767881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 769/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:59.436938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:59.438783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:59.451887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac440f1-f872-4dc2-ade6-88dad3b6c524 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:59.457212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:59.457853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:59.458028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:59.458364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] PATH_INFO: `/v3/projects/8a0f7a23b2854ea68ddf01c09a056d24` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:59.473356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0f7a23b2854ea68ddf01c09a056d24)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:59.474412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:16:59.474412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:16:59.474412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:16:59.475640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:59.643617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:59.682533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0f7a23b2854ea68ddf01c09a056d24', 'request_id': 'req-8bc4617c-465e-449c-91f0-2d7fa91d16d2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:59.689554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0f7a23b2854ea68ddf01c09a056d24', 'request_id': 'req-8bc4617c-465e-449c-91f0-2d7fa91d16d2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:59.799610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0f7a23b2854ea68ddf01c09a056d24', 'request_id': 'req-8bc4617c-465e-449c-91f0-2d7fa91d16d2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:59.819985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] The token cache is being invalidate because project 8a0f7a23b2854ea68ddf01c09a056d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:59.820417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:59.821102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8bc4617c-465e-449c-91f0-2d7fa91d16d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:59.822754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 741/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:59 2022] DELETE /identity/v3/projects/8a0f7a23b2854ea68ddf01c09a056d24 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:17:08.763012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=56,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:09.901685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:36.942597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:36.943318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:36.943610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:36.945869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:18:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:18:36.951216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333f441-17ad-4dac-b31f-6a01aecb2379 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:36.951701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333f441-17ad-4dac-b31f-6a01aecb2379 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:36.951873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333f441-17ad-4dac-b31f-6a01aecb2379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:36.993859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3333f441-17ad-4dac-b31f-6a01aecb2379 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:36.995083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3333f441-17ad-4dac-b31f-6a01aecb2379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:37.060017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:37.217947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:37.221919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:37.338121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc4966-2499-4963-a004-b35d89515b78 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:37.344257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:37.344932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:37.345243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:37.345516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:37.355628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:37.357327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3452ed-c94a-4210-97bf-1d3139a7f532 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:37.369071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 771/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:38.227725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b7a8-edcc-435c-abf3-11cf47c8cf8f None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:38.228320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b7a8-edcc-435c-abf3-11cf47c8cf8f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:38.228320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b7a8-edcc-435c-abf3-11cf47c8cf8f None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:38.229631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:38.234880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac22f1-8ab5-4fef-a9b8-71c04a8553d4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:38.235315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac22f1-8ab5-4fef-a9b8-71c04a8553d4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:38.235418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac22f1-8ab5-4fef-a9b8-71c04a8553d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:38.256921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-34ac22f1-8ab5-4fef-a9b8-71c04a8553d4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:38.258574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-34ac22f1-8ab5-4fef-a9b8-71c04a8553d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:38.268021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:38.423266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b7a8-edcc-435c-abf3-11cf47c8cf8f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:38.440303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b7a8-edcc-435c-abf3-11cf47c8cf8f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:38.446515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:38.447120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:38.447401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:38.447669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:38.458641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:38.460279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0570-c774-4926-bb74-a2e15ba9cd0b service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:38.470452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 744/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:39.070686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0dde4-a654-4995-9d66-d2f18b1b8f8b None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:39.070964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0dde4-a654-4995-9d66-d2f18b1b8f8b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:39.071235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0dde4-a654-4995-9d66-d2f18b1b8f8b None None] PATH_INFO: `/v3` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:39.072583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 773/1517] 10.222.0.76 () {54 vars in 925 bytes} [Mon Aug 29 19:18:39 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:18:39.079460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c65df4-fd3d-46e7-b1a2-f52a1aad2b69 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:39.079877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c65df4-fd3d-46e7-b1a2-f52a1aad2b69 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:39.080242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c65df4-fd3d-46e7-b1a2-f52a1aad2b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:39.137773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e2c65df4-fd3d-46e7-b1a2-f52a1aad2b69 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:39.138927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e2c65df4-fd3d-46e7-b1a2-f52a1aad2b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:39.221799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 745/1518] 10.222.0.76 () {58 vars in 1015 bytes} [Mon Aug 29 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:48.251343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=49 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:48.554202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89dc1-4058-49a8-8c79-999baffe4358 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:48.554455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89dc1-4058-49a8-8c79-999baffe4358 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:48.554625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89dc1-4058-49a8-8c79-999baffe4358 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:48.556106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 774/1519] 10.222.0.62 () {54 vars in 961 bytes} [Mon Aug 29 19:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:48.561598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fd63-e169-44c7-ab05-66eed1c0e538 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:48.562099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fd63-e169-44c7-ab05-66eed1c0e538 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:48.562206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fd63-e169-44c7-ab05-66eed1c0e538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:48.581987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9577fd63-e169-44c7-ab05-66eed1c0e538 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:48.583309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9577fd63-e169-44c7-ab05-66eed1c0e538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:48.592080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 746/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Aug 29 19:18:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:48.599641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89dc1-4058-49a8-8c79-999baffe4358 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:48.614214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89dc1-4058-49a8-8c79-999baffe4358 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:48.619888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:48.620509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:48.620789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:48.621061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:48.629802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:48.631540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb29a77-8b8d-45c6-b1ef-b2fb22270f63 service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:48.641886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 775/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:49.730842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243831f9-4e57-44fb-b013-9f0d6e4c39ec None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:49.731395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243831f9-4e57-44fb-b013-9f0d6e4c39ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:49.731395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243831f9-4e57-44fb-b013-9f0d6e4c39ec None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:49.733059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:49.738409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ae4a-00db-4a8c-bfba-806a4d0b4ebb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:49.738957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ae4a-00db-4a8c-bfba-806a4d0b4ebb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:49.738957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ae4a-00db-4a8c-bfba-806a4d0b4ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:49.759978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6031ae4a-00db-4a8c-bfba-806a4d0b4ebb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:49.761190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6031ae4a-00db-4a8c-bfba-806a4d0b4ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:49.769137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:18:49.929663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243831f9-4e57-44fb-b013-9f0d6e4c39ec None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:49.944794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243831f9-4e57-44fb-b013-9f0d6e4c39ec None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:49.950398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:49.951004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:49.951282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:49.951548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:49.960567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:49.962188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86f6e4-9f53-4bc6-b67f-f33b5a2eb9a7 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:49.972514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 748/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:50.815502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1573a0-8648-4c70-bc20-4d30e51506fb None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:50.816104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1573a0-8648-4c70-bc20-4d30e51506fb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:50.816104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1573a0-8648-4c70-bc20-4d30e51506fb None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:50.817586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:50.822573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce540e8-9c3c-4703-89ba-10e6ca3e8643 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:50.823044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce540e8-9c3c-4703-89ba-10e6ca3e8643 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:50.823153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce540e8-9c3c-4703-89ba-10e6ca3e8643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:50.842530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0ce540e8-9c3c-4703-89ba-10e6ca3e8643 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:50.843722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0ce540e8-9c3c-4703-89ba-10e6ca3e8643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:50.852082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:51.007805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1573a0-8648-4c70-bc20-4d30e51506fb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:51.023108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1573a0-8648-4c70-bc20-4d30e51506fb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:51.028853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:51.029496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:51.029773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:51.030139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:51.039433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:51.041099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba9a06-1f7e-40c8-84de-42ee95d44613 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:51.051111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 778/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:54.942633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:54.943231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:54.943231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:54.944543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:54.949814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd52c-6fda-4041-94c4-4749042b71b7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:54.950243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd52c-6fda-4041-94c4-4749042b71b7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:54.950352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd52c-6fda-4041-94c4-4749042b71b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:54.970970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f4acd52c-6fda-4041-94c4-4749042b71b7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:54.972247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f4acd52c-6fda-4041-94c4-4749042b71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:54.981009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:55.275865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:55.277599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:55.288964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:55.336394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1a554-fa80-4dce-998b-f4431b28d67c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:55.339999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:55.340609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:55.340887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:55.341164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:55.350382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:55.351844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e926a-675b-424f-88d8-e3fa65203b70 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:55.361686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 751/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:57.283587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bad02-9eed-4abc-9aea-f739a2dd952a None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:57.284166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bad02-9eed-4abc-9aea-f739a2dd952a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:57.284166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bad02-9eed-4abc-9aea-f739a2dd952a None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:57.285564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:18:57.291292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36bc2-11e3-445f-a418-b3966a76a5da None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:57.291765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36bc2-11e3-445f-a418-b3966a76a5da None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:57.291974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36bc2-11e3-445f-a418-b3966a76a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:57.313342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-40b36bc2-11e3-445f-a418-b3966a76a5da None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:57.314543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-40b36bc2-11e3-445f-a418-b3966a76a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:57.323040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:57.611382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bad02-9eed-4abc-9aea-f739a2dd952a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:57.623497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bad02-9eed-4abc-9aea-f739a2dd952a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:57.627173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:57.627788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:57.628110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:57.628353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:57.638292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:57.639751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396888da-b96a-4465-8c1a-11f0a08b8253 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:57.650047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 781/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:19:01.358299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:01.358816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:01.358816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:01.360347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:19:01.365938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52bad-5ddc-47da-905f-b83a6bf24f70 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:01.366429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52bad-5ddc-47da-905f-b83a6bf24f70 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:01.366541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52bad-5ddc-47da-905f-b83a6bf24f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:01.387219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-cbe52bad-5ddc-47da-905f-b83a6bf24f70 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:19:01.388446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cbe52bad-5ddc-47da-905f-b83a6bf24f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:01.396367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:19:01.956972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:01.958880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:02.018283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74ed79-467d-4bb6-a5ee-3c640a76fb16 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:02.021980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:02.022581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:02.022851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:02.023611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:02.032928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:02.034359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee6d9e-926a-4c11-836b-b47eb0d605cc service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:02.044379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 754/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:19:04.037931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:04.038557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:04.038557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:04.040447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:19:04.046027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7040-4a93-4acf-ba4e-4fbab6cf9141 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:04.046479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7040-4a93-4acf-ba4e-4fbab6cf9141 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:04.046606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7040-4a93-4acf-ba4e-4fbab6cf9141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:04.066113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e01d7040-4a93-4acf-ba4e-4fbab6cf9141 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:19:04.067357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e01d7040-4a93-4acf-ba4e-4fbab6cf9141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:04.075285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:19:04.637157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:04.638834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:04.652872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebead3-9013-4b26-a9fd-ac9f5de8d48d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:04.656968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:04.657840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:04.658417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:04.658826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:04.668821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:04.670263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a1097-d2fc-45e0-ad6e-93e28cfdf6e2 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:04.680350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 784/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:19:14.060665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=54 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:14.675600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=33 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:29.831352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:29.832070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:29.832070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:29.860474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:29.861719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:29.874425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 756/1540] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:29.887112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:29.887711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:29.887807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:29.918619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:29.925276 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:29.932259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 785/1541] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:29.937834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:29.953758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a73fa8-17ec-41f4-aebd-7ef9920c6536 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:29.961444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:29.962082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:29.962363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:29.962641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:29.965510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:29.965742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:29.967322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851c58-fd99-4c62-9f3a-c5dd307e976e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:29.991903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 757/1542] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.002305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.018243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cef04b-23ff-4168-8a5e-876c532ce27e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.026586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.028092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.028608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.029121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.029992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.032872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b028d8-6400-495e-b9ca-46045176a1b3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.110809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63d39b-cdaa-4312-88ed-7d4841a0c76e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.111007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63d39b-cdaa-4312-88ed-7d4841a0c76e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.111267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63d39b-cdaa-4312-88ed-7d4841a0c76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.138954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0d63d39b-cdaa-4312-88ed-7d4841a0c76e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.140251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0d63d39b-cdaa-4312-88ed-7d4841a0c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.152164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 758/1543] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.153858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a94c4-c6b1-49a9-9ed9-251fad935111 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.154340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a94c4-c6b1-49a9-9ed9-251fad935111 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.154829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a94c4-c6b1-49a9-9ed9-251fad935111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.176280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-123a94c4-c6b1-49a9-9ed9-251fad935111 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.177915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-123a94c4-c6b1-49a9-9ed9-251fad935111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.186509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 759/1544] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.188041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeeb33e-e302-430f-9d21-37232521241a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.188577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeeb33e-e302-430f-9d21-37232521241a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.189066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeeb33e-e302-430f-9d21-37232521241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.211777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-feeeb33e-e302-430f-9d21-37232521241a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.220005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-feeeb33e-e302-430f-9d21-37232521241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.221013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 786/1545] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:30.222421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.222774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.223269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.227781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 760/1546] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.229504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.229998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.230488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.254144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.256317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.259246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.260694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.324424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 787/1547] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.325331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.336755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db20eb-f235-42c0-a8be-7c59c368e319 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.342074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.342712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.342986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.343255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.343897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.344935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:30.344935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:20:30.344935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:20:30.346114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c47289-0570-4074-a277-68ed2ece83a5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.364933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 761/1548] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.366118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.379720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab2f72-64b5-40ac-87ce-eb20dc20bc1d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.384822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.385446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.385726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.385999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.386656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:30.386962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.388629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a4d80-ac6d-4219-8b34-ce34614bf4fe admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.400489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 762/1549] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.401679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ff9b-009e-4564-aca6-97b80e51f5c0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.401961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ff9b-009e-4564-aca6-97b80e51f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.402232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ff9b-009e-4564-aca6-97b80e51f5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.422362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-30a8ff9b-009e-4564-aca6-97b80e51f5c0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.423572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-30a8ff9b-009e-4564-aca6-97b80e51f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.435396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 763/1550] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.439691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.439691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.439691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.440042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 788/1551] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/users => generated 302 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:30.440042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fad03-f906-4e87-9f6d-20a489dacdda None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.440204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fad03-f906-4e87-9f6d-20a489dacdda None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.440626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fad03-f906-4e87-9f6d-20a489dacdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.457448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.458699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.462653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2c8fad03-f906-4e87-9f6d-20a489dacdda None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.463819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2c8fad03-f906-4e87-9f6d-20a489dacdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.466660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 764/1552] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.467489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.474753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 789/1553] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.476770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef062e0-f1bb-4e92-8fd6-c657bf37c9c8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.477351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef062e0-f1bb-4e92-8fd6-c657bf37c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.477787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef062e0-f1bb-4e92-8fd6-c657bf37c9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.478878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8e805-5965-40e9-b4c5-68bb5bf5c608 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.483830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.484466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.484755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.485159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.485818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:30.486122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.487800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.499416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2ef062e0-f1bb-4e92-8fd6-c657bf37c9c8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.500493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 765/1554] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.500765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ef062e0-f1bb-4e92-8fd6-c657bf37c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.501187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.508845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 790/1555] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.510066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.510369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.510644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.515497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e442c-da3a-40be-adda-626071da1214 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.520980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.521810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.521996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.522271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.522899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.523951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:30.523951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:20:30.523951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:20:30.525158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c096d-9429-4cab-8915-c90888b96f0e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.531226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.532471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.541225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 791/1556] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.542100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.552552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa90e-ee9c-42cc-b249-b98969f4ede9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.557265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.557870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.558149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.558423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.559079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.560711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e476261-37cf-4150-b617-fdf03368cd88 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.570852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 792/1557] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:30.572163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3c55-383f-4ab5-9541-cf56749db70b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.572789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3c55-383f-4ab5-9541-cf56749db70b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.572789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3c55-383f-4ab5-9541-cf56749db70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.591912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2b5c3c55-383f-4ab5-9541-cf56749db70b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.593167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2b5c3c55-383f-4ab5-9541-cf56749db70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.602396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 793/1558] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.604884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.604884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.604884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.624104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.625237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.633306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 794/1559] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.634078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.643647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8aef-42a4-4ffb-a7a1-154bc289cef5 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.649030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.654557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.654557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.654557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.654557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:30.654557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.655010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:30.655010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:20:30.655010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:20:30.655010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.665219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 795/1560] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.665981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.677560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929c2b-3961-4011-8110-ffb4be6b12ef admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.682452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.683015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.683289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.683557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.684249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.686158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.699585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 766/1561] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:30.702794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bd929-82c9-456d-b079-25c251d46972 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.702794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bd929-82c9-456d-b079-25c251d46972 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.702794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bd929-82c9-456d-b079-25c251d46972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.720710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b92bd929-82c9-456d-b079-25c251d46972 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.721904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b92bd929-82c9-456d-b079-25c251d46972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.772257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:30.780660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 767/1562] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.782119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.782585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.782904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.802265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:30.803471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:30.811886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 768/1563] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:30.812880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.813343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 796/1564] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:30.814174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.822891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5720d-d16e-425a-95b2-d5dfaf3fe12d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.846867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:30.878944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b539-9749-42f1-9f2b-f3efbcf8ffeb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.881137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.888743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.888743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.888743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.888743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.888743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.889261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.889261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.889261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.889261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.889928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:30.889928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.889928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.895972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 769/1565] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:30.896709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.903790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 797/1566] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.904474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.909894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251ad67-b444-4a53-8544-8fb6f8d79ad8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.914175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba4d67-0a31-41cf-995c-55ece19706c5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.914987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.915585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.915937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.916153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.916870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:30.917167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.918786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.918907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.919441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.919718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.920035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.920659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.922530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:30.931065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 770/1567] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:30.932052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:30.949267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07b83-7c41-47e5-becb-adb10d15573f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:30.954114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:30.954720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:30.955049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:30.955324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:30.956022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:30.958572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.098035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 771/1568] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.098903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.108703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907e41-e797-4190-8ed5-e52c704013cc admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.113242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.113832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.114123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.114395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.115040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:31.115339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.116971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.124393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 798/1569] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:30 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.124393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.137901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 772/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:31.137901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.143678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfcd4c-84c1-4b25-a5ec-838e9f1b2835 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.186865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075cfe7-f7dc-43bd-9dc9-b584fb99fc6a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.191997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.193166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.193450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.193729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.194673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:31.194673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.196391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.203285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.203790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.204092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.204365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.204990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.207423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.209269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 799/1571] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:31.210070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.225971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a2ca0-b4ec-419e-8798-7f35e5aeaf54 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.230755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.231412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.231688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.232001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] PATH_INFO: `/v3/projects/ef24d9e55c6e4cd1be40a094c6aa6422/users/de91a7723d93459b9c2521b064d1db64/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.247345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef24d9e55c6e4cd1be40a094c6aa6422, user_id=de91a7723d93459b9c2521b064d1db64, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.250031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.332558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 773/1572] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.333516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.343700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baed42-e82a-4f37-92cf-ad9e6976046d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.347821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.348526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.348800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.349088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.349862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.351979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.361404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 800/1573] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:31 2022] PUT /identity/v3/projects/ef24d9e55c6e4cd1be40a094c6aa6422/users/de91a7723d93459b9c2521b064d1db64/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:31.362303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.424089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314837b4-c06b-4829-9f77-47bf4bd242ff admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.429376 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.429980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.430258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.430530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.431150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.433089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.447575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:31.476780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 774/1574] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.477638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.491606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:31.539106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1718ddd-157c-496c-8ce8-5dc2648e2ef6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.543804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.544440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.544715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.544986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.545618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.547218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.557014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 801/1575] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.561666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.561795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 775/1576] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:31.561795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.570423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a07c3-8a7f-424b-af64-c8c992554f9c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.623348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b4f7-fd93-4a93-b4f0-0ea02716604a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.628540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.629151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.629362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.629842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.630132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.630411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.630713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.630948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.631410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.631562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.633379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.633541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.749563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 802/1577] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.750293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.759804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178cf7f-3fab-4901-a64f-b3e107bbf3fc admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.764508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.765106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.765379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.765646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.766266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.768189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.773600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 776/1578] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/projects => generated 367 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.774541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.834895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeabc2-c731-4556-a18e-7af1dd4f1fb0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.839545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.840183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.840454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.840721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.841349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.842973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b851c-eea2-4a9f-a130-4937a11bff9b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.853350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 777/1579] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:31.854607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.854941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.855237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.862640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 803/1580] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:31.863540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.880199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:31.881408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:31.910766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:31.933155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b2d80-8fa7-45e9-a8a9-c356357f2e2f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.938105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.938724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.939003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.939347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.939943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.941879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:31.946064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 778/1581] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:31.946840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:31.958390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbecb-65c8-4295-9bce-1372906c7eef None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:31.963545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:31.964215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:31.972518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:31.972518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:31.972518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:31.972518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.046698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 804/1582] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:32.047579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.057337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f386c-29cf-4ee4-b1ad-b2471a2c5f53 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.062028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.062626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.062904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.063176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.063812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.065432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.075415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 805/1583] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.076227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.086075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e6b5b-2719-4957-a650-7a0b6868a87a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.093045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 779/1584] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:31 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:32.093045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.093239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.093239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.093239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.093480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.093933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.095948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.100088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098dec1-2e22-443b-b811-1f9f299eb770 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.105239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.105851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.106128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.106607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.107237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.109644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 806/1585] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.109773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.110481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.123154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470fc83-6616-4a2e-99d8-3994eac8d84e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.127890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.128495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.132956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.132956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.132956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.132956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.141332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 807/1586] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.142204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.152215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645b495-ba9e-4fb8-9d45-f582c55611d7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.156832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.157533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.157813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.158083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.158704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.160632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.299089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 780/1587] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:32 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:32.299877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.310106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=11,keystone:SELECT=89 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:32.318071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 808/1588] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:32 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:32.318071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.325675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595f8e-50d8-4674-975e-1086a3b39093 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.330329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.330923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.331201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.331471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.332134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.334240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.344633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 809/1589] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.349149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.355745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cf3-9f3d-4782-9013-64bbfa130f31 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.358328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cddea4-3b88-4303-a7ee-46af46c4f39c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.360887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.361550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.361828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.362099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] PATH_INFO: `/v3/projects/f991d9be87e5494588cafcdcd6841e7d/users/f97d0910139b4610b792092b6133596d/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.362303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.362908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.363189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.363458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.373000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.374492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.377235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f991d9be87e5494588cafcdcd6841e7d, user_id=f97d0910139b4610b792092b6133596d, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.380614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.384691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 781/1590] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:32.385493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.395037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d2f0e-2b64-4e45-8a82-6a2c14b908b4 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.400879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.400879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.401294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.401657 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.402284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.403895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.417433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 782/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.418326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.428584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146d996-c270-4e8e-95ac-a20210b3073e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.433298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.436637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.436637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.436637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.436637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.437020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.447669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 783/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.448536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.458359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd6ce-8bf3-4a96-a9b8-88b1b9f4f9fa admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.462868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.463513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.463791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.471033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] PATH_INFO: `/v3/projects/654eea424bd24f2bb447382c435f2c1d/users/00c1651cd24b4bb2ac09945a16aba093/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.478868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654eea424bd24f2bb447382c435f2c1d, user_id=00c1651cd24b4bb2ac09945a16aba093, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.479911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:32.479911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:20:32.479911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:20:32.481853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.509935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 810/1593] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:32 2022] PUT /identity/v3/projects/f991d9be87e5494588cafcdcd6841e7d/users/f97d0910139b4610b792092b6133596d/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:32.510759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.574634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b912eb-3072-4800-b273-a2a317a90cf8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.580100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.580421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.580702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.580973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.581612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.583827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.594113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 811/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.595089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.605200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aacdc-8ed4-4cb7-b8f8-d701be16c42b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.609101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 784/1595] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:32 2022] PUT /identity/v3/projects/654eea424bd24f2bb447382c435f2c1d/users/00c1651cd24b4bb2ac09945a16aba093/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:32.610055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.665010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:32.672006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.672621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.672984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.673298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.673962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.675567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.675864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be401c70-0e3a-4094-b3f4-fafeb9123738 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.680619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.685005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.685005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.685005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.685005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.685005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.685730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 812/1596] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.686549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.695068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 785/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.696068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.698801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa0f51-1177-40ce-954b-6d12e88867ed admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.703507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.704144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.704641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.704777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.705308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.705930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c0ee9-a9fe-4533-b445-250ddae99e11 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.706888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e8203-399b-42e5-8cb1-08e8aa0b46a2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.710820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.711464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.711742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.712051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] PATH_INFO: `/v3/projects/76a7037e22b543268635379f8491c774/users/1c2b79cbc9fb4751a51986d9cf8ff5ad/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.717651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 813/1598] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.718864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.725746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.725746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.729837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a7037e22b543268635379f8491c774, user_id=1c2b79cbc9fb4751a51986d9cf8ff5ad, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.732664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.738626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:32.739974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:32.747906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 814/1599] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:32.753879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.760370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224beb-29b7-46af-b65a-5443048c93d2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.765638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.766266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.766543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.766810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.767478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.769163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfea0cc-fad6-4769-b1ec-df55d314857a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.780913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 815/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.780913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.780913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.780913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.815996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] MFA Rules not processed for user `00c1651cd24b4bb2ac09945a16aba093`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:32.817885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:32.835431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:32.874383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 786/1601] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:32 2022] PUT /identity/v3/projects/76a7037e22b543268635379f8491c774/users/1c2b79cbc9fb4751a51986d9cf8ff5ad/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:32.874383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.896433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 816/1602] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:32.897294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.910270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189b07-1e58-4b3d-8724-e5a108ef7913 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.950932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911434-4465-4e46-bf38-5efe738c1dc2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.957251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.966198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.966198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.966198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.966198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:32.966198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:32.972152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 787/1603] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:32.973601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:32.975885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.976732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.977013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.977291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] PATH_INFO: `/v3/projects/de427c8638c049d3ad809158b0ccc0d2/users/74bfde91bf944824bdae8207641a08b7/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:32.984901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf800a0-f049-4f84-9891-86dceee9a1ab admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:32.990478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:32.991286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:32.991413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:32.991628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] PATH_INFO: `/v3/projects/837fdaee5fde423587a1418f8f322ff2/users/a7b0a420e50e49fe87365540ce9bf30d/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.009761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=837fdaee5fde423587a1418f8f322ff2, user_id=a7b0a420e50e49fe87365540ce9bf30d, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.013364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.026791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de427c8638c049d3ad809158b0ccc0d2, user_id=74bfde91bf944824bdae8207641a08b7, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.029746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.195940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 788/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:32 2022] PUT /identity/v3/projects/837fdaee5fde423587a1418f8f322ff2/users/a7b0a420e50e49fe87365540ce9bf30d/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:33.202283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 817/1605] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:32 2022] PUT /identity/v3/projects/de427c8638c049d3ad809158b0ccc0d2/users/74bfde91bf944824bdae8207641a08b7/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:33.203345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:33.208249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:33.210042 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.270510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5549-c592-4b4e-a3bc-2fa00b040e4e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:33.271906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae16bc-c883-4ba9-9887-92c32514d969 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:33.276270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:33.276878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.277152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.277418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.320617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:33.329559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:33.336230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:33.336230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.336230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.336810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] PATH_INFO: `/v3/projects/29fc03040cd14f6cb4246bef42ae2166/users/0fc03a7273624d98bdfb4b7846c3a78f/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.344250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.345966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6171368-a620-411b-8b84-73fa162d9628 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.353737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fc03040cd14f6cb4246bef42ae2166, user_id=0fc03a7273624d98bdfb4b7846c3a78f, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.356384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.357180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 789/1606] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:33.358699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.358988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.359277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.389010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] MFA Rules not processed for user `1c2b79cbc9fb4751a51986d9cf8ff5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:33.390234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.424478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 818/1607] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:33 2022] PUT /identity/v3/projects/29fc03040cd14f6cb4246bef42ae2166/users/0fc03a7273624d98bdfb4b7846c3a78f/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:33.436987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:33.469536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 790/1608] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:33.470974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:33.483357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7c8fb-8582-436c-bf26-31349dc6c57d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:33.514949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dc5c3-0265-474d-ae4f-2210c75cceeb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:33.520798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:33.521520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.521858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.522189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.522930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.525277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79de01-8011-4d1a-b156-cee0a656f957 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.555239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:33.555931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.556206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.556480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] PATH_INFO: `/v3/projects/fec9a23227784d23ba703719c785e413/users/bd993c0493d447cb9b02e424a28e24bc/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.572362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec9a23227784d23ba703719c785e413, user_id=bd993c0493d447cb9b02e424a28e24bc, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.575060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d3500-1c8f-41a4-99a6-aafb26a46961 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.693877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 819/1609] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:33.693877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ea284-1452-41f1-a9e2-63751d6c9705 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.693877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ea284-1452-41f1-a9e2-63751d6c9705 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.693877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ea284-1452-41f1-a9e2-63751d6c9705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.717857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b09ea284-1452-41f1-a9e2-63751d6c9705 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:33.719380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b09ea284-1452-41f1-a9e2-63751d6c9705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.727525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 791/1610] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:33 2022] PUT /identity/v3/projects/fec9a23227784d23ba703719c785e413/users/bd993c0493d447cb9b02e424a28e24bc/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:33.729711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ea00-5263-4ffa-824c-053cffa4d859 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.729711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ea00-5263-4ffa-824c-053cffa4d859 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.729711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ea00-5263-4ffa-824c-053cffa4d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.763921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-59b9ea00-5263-4ffa-824c-053cffa4d859 None None] MFA Rules not processed for user `a7b0a420e50e49fe87365540ce9bf30d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:33.765291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-59b9ea00-5263-4ffa-824c-053cffa4d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.792269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 820/1611] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:33.793818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addcd18-a9c1-4495-88e0-8311f1effec4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.794172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addcd18-a9c1-4495-88e0-8311f1effec4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.794516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addcd18-a9c1-4495-88e0-8311f1effec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.826622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5addcd18-a9c1-4495-88e0-8311f1effec4 None None] MFA Rules not processed for user `0fc03a7273624d98bdfb4b7846c3a78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:33.828145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5addcd18-a9c1-4495-88e0-8311f1effec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.830920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:33.851107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 792/1612] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:33.852481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a063d-1b71-4ccc-84bd-52946e3cad4b None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.852762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a063d-1b71-4ccc-84bd-52946e3cad4b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.853035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a063d-1b71-4ccc-84bd-52946e3cad4b None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.854450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 793/1613] 10.222.0.62 () {54 vars in 962 bytes} [Mon Aug 29 19:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:33.857197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a063d-1b71-4ccc-84bd-52946e3cad4b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:33.876677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a063d-1b71-4ccc-84bd-52946e3cad4b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:33.882666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:33.883323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.883726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.884030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.884735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:33.886648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f227a-2a8e-4476-9def-b868ee2b8159 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:33.889775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:33.914383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 821/1614] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:33.915808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:33.916186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:33.916465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:33.936705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:33.938257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:33.947013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 822/1615] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:33.947982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.018268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dfcdd-4ec1-40b1-8c61-29d991d0c274 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.022671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.023289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.023595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.023845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.033585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.035029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d228062-ba76-4adf-a66a-b1be2317ba78 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.045876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 823/1616] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:34.049314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.049752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.050098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.062495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 794/1617] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:33 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:34.064888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a57-7416-40f2-986c-d07a34746e3c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.067352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a57-7416-40f2-986c-d07a34746e3c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.067779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a57-7416-40f2-986c-d07a34746e3c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.069387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 795/1618] 10.222.0.62 () {54 vars in 962 bytes} [Mon Aug 29 19:20:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:34.071410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a57-7416-40f2-986c-d07a34746e3c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.113322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:34.115348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:34.153681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a57-7416-40f2-986c-d07a34746e3c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.158176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.159031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.163994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.164349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.175466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.178178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.187726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 796/1619] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:34.189006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.196903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 824/1620] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:34.198325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.207220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb468ae-01f1-4266-bcd7-82f31adb2a4b service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.211202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1454af-27c0-4208-b258-0f37834acaee None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.211822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.212595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.212734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.213011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.218310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.220337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.220618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.220891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.221543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.223931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105876-5a9e-4adb-83d4-4f252a1edf7d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.226251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.228388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e3657-c0b4-4f13-b376-34b94b7ded8f service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.235295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 825/1621] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:34.236698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.236978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.237250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.246512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 797/1622] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:34.247617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.250763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.250763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.273951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:34.275515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:34.286087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 826/1623] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:34.287404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.289420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] MFA Rules not processed for user `de91a7723d93459b9c2521b064d1db64`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:34.291118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:34.300212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045ce09-e5c9-408a-8e24-f380414914f6 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.312334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.313985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.314521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.314892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.370948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 798/1624] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:34.371745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.383697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed92d09-9ba3-4150-abf7-5453145c5f76 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.389462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.390141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.390419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.390695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] PATH_INFO: `/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38/users/5a3b6976eb684fbf8c28f60fed2b1464/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.394344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.397472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.408054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3a9d4b88954972a23c0e8d2f306c38, user_id=5a3b6976eb684fbf8c28f60fed2b1464, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.410873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 827/1625] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:34.411160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.412153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.425874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02a153-b6b4-48b2-8fcd-85e1fe18ccf0 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.433744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.434825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.435230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.435924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.448129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.450149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.465814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 828/1626] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:34.467010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.484355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89907-e5be-4f71-b446-683db78f477e service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.498632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.502467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.504270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.506384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.508005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.512705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ce4da-805a-4f19-a132-d1cf8ec085a1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.533028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 799/1627] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:34 2022] PUT /identity/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38/users/5a3b6976eb684fbf8c28f60fed2b1464/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:34.536907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.625299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776da44-699c-4e6a-90b8-e99e2d30a012 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.631109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.631870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.632157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:34.632677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:34.633382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:34.635179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:34.648914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 800/1628] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:34.654819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:34.685449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b8457-9990-4aa6-8f5a-5ba58c8fa791 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:34.703001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:34.703951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:34.704376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.030084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] PATH_INFO: `/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38/users/5a3b6976eb684fbf8c28f60fed2b1464/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.030084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3a9d4b88954972a23c0e8d2f306c38, user_id=5a3b6976eb684fbf8c28f60fed2b1464, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.030084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819f4d-7297-4526-87dc-549bd4624a99 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.030084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 801/1629] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:34 2022] PUT /identity/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38/users/5a3b6976eb684fbf8c28f60fed2b1464/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:35.030084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc05586-627f-435e-b068-2e9f171ce4ad None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.031440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc05586-627f-435e-b068-2e9f171ce4ad None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.031440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc05586-627f-435e-b068-2e9f171ce4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.031440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ccc05586-627f-435e-b068-2e9f171ce4ad None None] MFA Rules not processed for user `5a3b6976eb684fbf8c28f60fed2b1464`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:35.031440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ccc05586-627f-435e-b068-2e9f171ce4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 829/1630] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/projects => generated 405 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e84b-3927-45a1-9d95-50bff0071fba None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e84b-3927-45a1-9d95-50bff0071fba None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e84b-3927-45a1-9d95-50bff0071fba None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 830/1631] 10.222.0.62 () {54 vars in 912 bytes} [Mon Aug 29 19:20:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:35.032233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e84b-3927-45a1-9d95-50bff0071fba None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.033779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 802/1632] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:35.033779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1451a2-68a1-479a-8a05-6c2b6f7f9e43 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.033779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1451a2-68a1-479a-8a05-6c2b6f7f9e43 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.033779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1451a2-68a1-479a-8a05-6c2b6f7f9e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.034705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e84b-3927-45a1-9d95-50bff0071fba None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.034705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.034705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.034705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.035204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.035204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.035671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ee1451a2-68a1-479a-8a05-6c2b6f7f9e43 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:35.035671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ee1451a2-68a1-479a-8a05-6c2b6f7f9e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:35.036748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 803/1633] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3d3b6-9c98-4900-a0cd-b4d94c2f56f8 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3d3b6-9c98-4900-a0cd-b4d94c2f56f8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3d3b6-9c98-4900-a0cd-b4d94c2f56f8 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 804/1634] 10.222.0.62 () {54 vars in 912 bytes} [Mon Aug 29 19:20:34 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:35.036821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3d3b6-9c98-4900-a0cd-b4d94c2f56f8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.055402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3d3b6-9c98-4900-a0cd-b4d94c2f56f8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.059706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.060526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.060801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.061172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.138483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.140162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ddee-c9df-44c3-97d2-3cd6d5af9849 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.152671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 805/1635] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:35.153820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.154205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.154376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.176464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:35.177891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:35.188631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 806/1636] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:20:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:35.320150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 831/1637] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:34 2022] POST /identity/v3/users => generated 309 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:35.324441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.336730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1382d4e-8c92-49df-8d16-4ba8ebeec938 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.341379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.342196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.342342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.342537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.346284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.350847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.366857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.366970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 807/1638] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:35.367043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.374661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042ed4-376a-43e5-917f-297834871753 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.377957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52bec-4d51-4f42-aa91-a32dec8df644 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.379362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.381524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.381524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.381524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.383916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.389545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.389545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.389545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] PATH_INFO: `/v3/projects/b634224115b042b68696f82f0ab99ff9/users/c5cd8ccb1d844fdcb41fd89dbe646b80/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.396358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.397940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.403286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b634224115b042b68696f82f0ab99ff9, user_id=c5cd8ccb1d844fdcb41fd89dbe646b80, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.406227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.413171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 832/1639] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:35.473857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.503643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412dc1f-fae5-4b34-bbbd-b6c3411f3d18 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.519218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.519958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.520257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.520529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.534190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.534893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:35.541663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.550287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 833/1640] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:20:35.623347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 808/1641] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:35 2022] PUT /identity/v3/projects/b634224115b042b68696f82f0ab99ff9/users/c5cd8ccb1d844fdcb41fd89dbe646b80/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:35.624063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.639556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:35.646106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22f582-e47f-483d-9d20-4e69eb822cf7 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.718547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.718547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.718547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.718547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.718914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.719145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.734065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 834/1642] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:35.735228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:35.746959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672532ae-dc16-4cb7-b026-871362f97456 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:35.753847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:35.755093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:35.755463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:35.756062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] PATH_INFO: `/v3/projects/b634224115b042b68696f82f0ab99ff9/users/c5cd8ccb1d844fdcb41fd89dbe646b80/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:35.758541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b634224115b042b68696f82f0ab99ff9, user_id=c5cd8ccb1d844fdcb41fd89dbe646b80, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:35.761467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:35.782775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 809/1643] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:35 2022] PUT /identity/v3/projects/b634224115b042b68696f82f0ab99ff9/users/c5cd8ccb1d844fdcb41fd89dbe646b80/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:20:36.138613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:36.238247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:36.241401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e242974-1d9a-43e0-8763-5f25395e8e06 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:36.303661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:36.304502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:36.305095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:36.305374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:36.400044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:36.403086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d814-d022-422a-b660-300d38fed98f service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:36.427926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 835/1644] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:36.868566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:36.932771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:37.018282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087145-a01e-4d61-be9d-58a08512ed48 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:37.043068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:37.043662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:37.043954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:37.044225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] PATH_INFO: `/v3/users/1c2b79cbc9fb4751a51986d9cf8ff5ad` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:37.047948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2b79cbc9fb4751a51986d9cf8ff5ad)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:37.055266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: WARNING py.warnings [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:37.055266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: warnings.warn(msg) Aug 29 19:20:37.055266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: Aug 29 19:20:37.056700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:37.432360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2b79cbc9fb4751a51986d9cf8ff5ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:37.591311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2b79cbc9fb4751a51986d9cf8ff5ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:37.592048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2b79cbc9fb4751a51986d9cf8ff5ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:37.606995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2b79cbc9fb4751a51986d9cf8ff5ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:37.649541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2b79cbc9fb4751a51986d9cf8ff5ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:37.666290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 810/1645] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:36 2022] DELETE /identity/v3/users/1c2b79cbc9fb4751a51986d9cf8ff5ad => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:37.671471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee9a3-d9d7-4b6a-a239-c3cae2eabe2d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:37.671922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee9a3-d9d7-4b6a-a239-c3cae2eabe2d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:37.672134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee9a3-d9d7-4b6a-a239-c3cae2eabe2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:37.698788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5c9ee9a3-d9d7-4b6a-a239-c3cae2eabe2d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:37.700214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5c9ee9a3-d9d7-4b6a-a239-c3cae2eabe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:37.792812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 836/1646] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:37.821935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:37.824273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:37.950756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31fa6e-40a8-47ca-8721-b9d1a8e12a61 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:37.957572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:37.959702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:37.960359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:37.967259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:37.979612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:37.981571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:38.005630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 811/1647] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:39.120901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270240be-f7f6-4c5c-baf2-09fa28cee921 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:39.121366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270240be-f7f6-4c5c-baf2-09fa28cee921 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:39.122121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270240be-f7f6-4c5c-baf2-09fa28cee921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:39.192218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-270240be-f7f6-4c5c-baf2-09fa28cee921 None None] MFA Rules not processed for user `c5cd8ccb1d844fdcb41fd89dbe646b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:39.194227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-270240be-f7f6-4c5c-baf2-09fa28cee921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:39.203292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:39.335131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 837/1648] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:39.794773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:39.803393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:39.845328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751d9-2780-4e1d-a091-d0b34c831885 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:39.865649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:39.866316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:39.871178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:39.871178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] PATH_INFO: `/v3/projects/76a7037e22b543268635379f8491c774` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:39.876907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a7037e22b543268635379f8491c774)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:39.880296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:40.745487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:40.846780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a7037e22b543268635379f8491c774', 'request_id': 'req-578b558f-a08c-4733-8d1c-decfeb04b43f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:40.857678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a7037e22b543268635379f8491c774', 'request_id': 'req-578b558f-a08c-4733-8d1c-decfeb04b43f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:40.984284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a7037e22b543268635379f8491c774', 'request_id': 'req-578b558f-a08c-4733-8d1c-decfeb04b43f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.038483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] The token cache is being invalidate because project 76a7037e22b543268635379f8491c774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:41.039094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.040049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-578b558f-a08c-4733-8d1c-decfeb04b43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.043618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 812/1649] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:39 2022] DELETE /identity/v3/projects/76a7037e22b543268635379f8491c774 => generated 0 bytes in 1250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:41.049316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:41.049787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:41.050005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:41.096589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:41.099407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:41.332082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 838/1650] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:41.341128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:41.341128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:41.341291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:41.355380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:41.358544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:41.387563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde89fa-f9f5-4ed3-8d3f-91eb67bae8c1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:41.413469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:41.418623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:41.420258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:41.420949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:41.421223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:41.421493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] PATH_INFO: `/v3/users/a7b0a420e50e49fe87365540ce9bf30d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:41.427996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b0a420e50e49fe87365540ce9bf30d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:41.431953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:41.446071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 813/1651] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:41.447252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:41.484571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e8e6e-0801-4a2e-8585-b055ed35c950 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:41.517516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:41.522224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:41.557570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 814/1652] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:41.563403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:41.590297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6d45-79dc-49fb-9938-8c86aa0a14f8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:41.612655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:41.613283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:41.613644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:41.614521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:41.615158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:41.617115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:41.821963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 815/1653] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:41 2022] POST /identity/v3/projects => generated 399 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:41.823396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:41.844493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b0a420e50e49fe87365540ce9bf30d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.846325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b0a420e50e49fe87365540ce9bf30d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.896683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b0a420e50e49fe87365540ce9bf30d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:41.928757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b0a420e50e49fe87365540ce9bf30d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:42.011066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deb609-ac0a-4aeb-87c0-3f20d987f71a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.032715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.034516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.034904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.035879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.048670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.051567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:42.106679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2cac2e2e-b215-4ba5-a534-83b56ed4f540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b0a420e50e49fe87365540ce9bf30d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:42.115561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 839/1654] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:41 2022] DELETE /identity/v3/users/a7b0a420e50e49fe87365540ce9bf30d => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:42.120116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c2731-bb98-4b0e-b3c9-eb43cfb4f197 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.120799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c2731-bb98-4b0e-b3c9-eb43cfb4f197 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.121298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c2731-bb98-4b0e-b3c9-eb43cfb4f197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.170828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-443c2731-bb98-4b0e-b3c9-eb43cfb4f197 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:42.180012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-443c2731-bb98-4b0e-b3c9-eb43cfb4f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:42.214341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:42.247379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 816/1655] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:41 2022] POST /identity/v3/users => generated 300 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:42.248706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:42.255723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:42.321027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a4da-7ff3-4fb5-8735-a31bda021956 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.399306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 840/1656] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:42.400795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442308-d1ea-46d4-8a12-c5d87fd8eb59 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.401081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442308-d1ea-46d4-8a12-c5d87fd8eb59 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.401357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442308-d1ea-46d4-8a12-c5d87fd8eb59 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.414116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 841/1657] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:20:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:42.458729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.459014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.459286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.517301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:42.518662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:42.538390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.541002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.541002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.541002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.542548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.548176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:42.580318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 817/1658] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:42.581578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:42.584614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:42.661779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535c8f-37e0-4a2f-8c0a-0120ceaa1431 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.689616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.690508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.696434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.696871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.706866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 842/1659] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:42.706866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:42.717378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.727761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:42.746646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa538a5-a26b-43ad-89da-926e5c7aab7f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.757794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.758438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.758846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.759125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.759886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.762592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 818/1660] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:42.763110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:42.763926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:42.767248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:42.790096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6c350-28eb-49a5-afff-af7b8e41c88d service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.793033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 843/1661] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:42.796315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:42.803294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.804562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.805033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.805822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.827036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04d3b-074d-48ea-bc82-0bff35f9704c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:42.838447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.841127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4796e1-5302-48cc-874a-559a23fcc505 service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:42.850552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:42.851173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:42.851823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:42.851823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] PATH_INFO: `/v3/projects/11820555aa474e98954000a1238ece79/users/8784201e56be42529494c91dc566c898/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:42.885024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 819/1662] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:42.890189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11820555aa474e98954000a1238ece79, user_id=8784201e56be42529494c91dc566c898, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:42.898443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:43.003528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 844/1663] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:42 2022] PUT /identity/v3/projects/11820555aa474e98954000a1238ece79/users/8784201e56be42529494c91dc566c898/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:43.005627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:43.006274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:43.008273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:43.140446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] MFA Rules not processed for user `8784201e56be42529494c91dc566c898`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:43.152624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:43.344750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 820/1664] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:43.345091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:43.525098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4d079-fa25-4ba7-a2ca-e9f12bb2ae0e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:43.542149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:43.543107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:43.544258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:43.544706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:43.545500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:43.556493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:43.774076 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 845/1665] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:43 2022] POST /identity/v3/projects => generated 401 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:20:43.778943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:43.807047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454db91-9849-4c5a-9cb5-b7d8d718ebd8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:43.950999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:43.952019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:43.952511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:43.953004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:43.953853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:43.956385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.018203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:44.093463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 821/1666] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:43 2022] POST /identity/v3/users => generated 307 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:44.097084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:44.112714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48b458-5687-45b4-b506-4667ace9f611 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:44.127782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:44.128592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.129033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.129506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.132917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:44.137291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.165981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 846/1667] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:44.169382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:44.184996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519867-d4bb-4215-b5df-67e9b4c5790f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:44.201159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:44.201939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.202291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.202638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] PATH_INFO: `/v3/projects/13186aa95168469c90ec357870d80687/users/558042483ae74198bce7e1306ceb4d9c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.241559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13186aa95168469c90ec357870d80687, user_id=558042483ae74198bce7e1306ceb4d9c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:44.244323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.419378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 822/1668] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:44 2022] PUT /identity/v3/projects/13186aa95168469c90ec357870d80687/users/558042483ae74198bce7e1306ceb4d9c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:44.427704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:44.445379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a39504-a5ba-429e-8525-bfaf8d106392 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:44.554266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:44.557398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:44.599271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:44.599933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.600215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.600487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.601138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:44.602794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.637642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 847/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:44.640904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:44.671499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d5e5-9f55-40aa-a463-08acc87858db admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:44.687353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:44.688463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.688934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.689355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] PATH_INFO: `/v3/projects/13186aa95168469c90ec357870d80687/users/558042483ae74198bce7e1306ceb4d9c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.692651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13186aa95168469c90ec357870d80687, user_id=558042483ae74198bce7e1306ceb4d9c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:44.696082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90ad4-87af-4c8a-804d-f92498ecd37a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.706335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764e63-05ed-4ebe-ae5e-3ee0bdb2bc18 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:44.742536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 848/1670] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:44 2022] PUT /identity/v3/projects/13186aa95168469c90ec357870d80687/users/558042483ae74198bce7e1306ceb4d9c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:44.752682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.753154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.753726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.804861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:44.805927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:44.806375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:44.806815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] PATH_INFO: `/v3/projects/837fdaee5fde423587a1418f8f322ff2` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:44.809824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=837fdaee5fde423587a1418f8f322ff2)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:44.812534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:44.814829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] MFA Rules not processed for user `558042483ae74198bce7e1306ceb4d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:44.817328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:44.896137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:44.959803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '837fdaee5fde423587a1418f8f322ff2', 'request_id': 'req-47400eb9-c5e2-43ca-a75c-befd58e318d9'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:44.968287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '837fdaee5fde423587a1418f8f322ff2', 'request_id': 'req-47400eb9-c5e2-43ca-a75c-befd58e318d9'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:45.036161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 849/1671] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:44 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:45.054696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:45.056266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.178589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '837fdaee5fde423587a1418f8f322ff2', 'request_id': 'req-47400eb9-c5e2-43ca-a75c-befd58e318d9'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:45.200343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] The token cache is being invalidate because project 837fdaee5fde423587a1418f8f322ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:45.201337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:45.203336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47400eb9-c5e2-43ca-a75c-befd58e318d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:45.208640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 823/1672] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:44 2022] DELETE /identity/v3/projects/837fdaee5fde423587a1418f8f322ff2 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:20:45.212337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8863c5-d869-43d2-ac51-731107b0867f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:45.219548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb271-5ca5-40cb-8000-748bdf551fd8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.219902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb271-5ca5-40cb-8000-748bdf551fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.220160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb271-5ca5-40cb-8000-748bdf551fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.291834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-50ffb271-5ca5-40cb-8000-748bdf551fd8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:45.294324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-50ffb271-5ca5-40cb-8000-748bdf551fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.376126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:45.382851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.382851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.382851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.382851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.471833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:45.474644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:45.484750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 824/1673] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:45.485718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 850/1674] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:45 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:45.502832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611c88-5e72-4768-a759-fa31b028ecec None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.503586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611c88-5e72-4768-a759-fa31b028ecec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.504542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611c88-5e72-4768-a759-fa31b028ecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.538076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-60611c88-5e72-4768-a759-fa31b028ecec None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:45.539524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-60611c88-5e72-4768-a759-fa31b028ecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.554283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:45.560255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 825/1675] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:45.576868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8921e-a27b-4692-ac20-32e2b776118e service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:45.588648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e691-1e40-45f1-9295-00bd8ddf0b0b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.589047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e691-1e40-45f1-9295-00bd8ddf0b0b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.589449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e691-1e40-45f1-9295-00bd8ddf0b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.599167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:45.600045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.600477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.600908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.601564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:45.601866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:45.603487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:45.639002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 851/1676] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:45.640095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:45.649790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c2f7e691-1e40-45f1-9295-00bd8ddf0b0b None None] MFA Rules not processed for user `f97d0910139b4610b792092b6133596d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:45.651761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c2f7e691-1e40-45f1-9295-00bd8ddf0b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.656141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9def4c-6144-48b6-aa62-1b5c41210768 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:45.664892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:45.665684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.666071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.666785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.667557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:45.669930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:45.748216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 826/1677] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:45.751650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.752022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.761177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.779100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] MFA Rules not processed for user `f97d0910139b4610b792092b6133596d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:45.780402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.790167 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 852/1679] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:45.792038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 827/1679] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:45.792164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:45.792967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.796543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:45.818981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a60a6c-04a3-4977-9d4c-4a166cc82428 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:45.930314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a512-7c34-4d3a-8ca0-baffdc214bd6 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:45.943571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:45.944398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.944793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.945213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.947349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:45.979445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:45.980109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:45.980386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:45.980658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:45.982210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:45.984462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75d51-d5bb-4d47-9cc8-ebe3c10f6426 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.065550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.067775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.078640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 853/1680] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:45 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:46.079548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.081028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:46.175017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 828/1681] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:45 2022] POST /identity/v3/users => generated 288 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:46.176563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677774a-c5e7-4d02-9c7d-157ad57a273e None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.176953 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677774a-c5e7-4d02-9c7d-157ad57a273e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.177328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677774a-c5e7-4d02-9c7d-157ad57a273e None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.178689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 829/1682] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:46.193097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677774a-c5e7-4d02-9c7d-157ad57a273e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.221150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677774a-c5e7-4d02-9c7d-157ad57a273e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.229311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6a57-c578-498b-be59-cd87cde92693 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.237759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.240976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.240976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.240976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.241211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.242768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.244005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.244503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.246758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.248889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992d3fd-e19e-425f-b123-2411a1498d15 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.264965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 830/1683] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:46.266756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.269819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.269819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.299480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:46.302021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:46.356762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.358957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.394099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 854/1684] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:46.397484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.412101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:46.415326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 831/1685] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:46.432538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.434931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a041ec-8aef-4b76-9436-0b4976a7d17d service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.450346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.452223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.452223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.452223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] PATH_INFO: `/v3/users/de91a7723d93459b9c2521b064d1db64` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.454098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de91a7723d93459b9c2521b064d1db64)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.457567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.466900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4120c7-f60f-4974-86b5-2b8f5c3497e7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.488152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.489900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.490181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.491668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.494918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.503142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.563163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 832/1686] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:46.564402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.570094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:46.605124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf838b5-80e5-476d-9a86-a5ed2d2e28da admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.621704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.622370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.622646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.622917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.643233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.645440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.652251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:46.678159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 833/1687] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:46.682365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:46.708884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d99ab9-8d4a-4540-bccc-c5f2bcf4e670 service placement] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:46.717667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:46.718340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.718636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.718878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] PATH_INFO: `/v3/projects/8a85bff1226645af8921164ad515a506/users/1eb26f9e1b28484fb3216feecba34b56/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.764881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a85bff1226645af8921164ad515a506, user_id=1eb26f9e1b28484fb3216feecba34b56, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:46.768047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24ced0-693f-4b5e-8508-b6211f3a960e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:46.861860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:46.889483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de91a7723d93459b9c2521b064d1db64'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:46.900120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 834/1688] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:46 2022] PUT /identity/v3/projects/8a85bff1226645af8921164ad515a506/users/1eb26f9e1b28484fb3216feecba34b56/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:46.912412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de91a7723d93459b9c2521b064d1db64'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:46.912708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:46.913181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:46.913644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:46.959688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de91a7723d93459b9c2521b064d1db64'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:46.959877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] MFA Rules not processed for user `1eb26f9e1b28484fb3216feecba34b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:46.961178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:46.981984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de91a7723d93459b9c2521b064d1db64'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:47.105221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 835/1689] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:47.107337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:47.107796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-16df816f-8eae-4152-9bd4-04cc1f43ed38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de91a7723d93459b9c2521b064d1db64'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:47.134597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 855/1690] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:46 2022] DELETE /identity/v3/users/de91a7723d93459b9c2521b064d1db64 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:47.202647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680175f-0c01-4157-af6a-db23431a9c96 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.202987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680175f-0c01-4157-af6a-db23431a9c96 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.203647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680175f-0c01-4157-af6a-db23431a9c96 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.205494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 856/1691] 10.222.0.62 () {54 vars in 912 bytes} [Mon Aug 29 19:20:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:47.215366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.216474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.223923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.274600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6c0ba-389c-4cc2-a9e7-5af91487c91e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:47.326104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:47.333476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:47.410535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:47.410535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.410535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.410535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.411404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:47.411404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:47.434973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 857/1692] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:20:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:47.452240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:47.454236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:47.467389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17467d-93a9-4f07-9b2c-01d3478d9902 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:47.471455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:47.472177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.472461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.472734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.503922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:47.508878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:47.532326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 858/1693] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:20:47.578430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 836/1694] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:47 2022] POST /identity/v3/projects => generated 375 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:47.578599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:47.578704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:47.607720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acce14-4caf-40ae-bbc1-350dfb2348f0 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:47.813592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:47.813592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.813592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.813592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.814597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:47.814597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:47.960085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 859/1695] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:47 2022] POST /identity/v3/users => generated 294 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:47.965012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:47.980450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945c49-d320-45c7-a173-24f7e3f3063e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:47.994310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:47.995129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:47.995826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:47.996840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:47.997707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.000399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.016483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 837/1696] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:20:48.018823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.031644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1019e-2601-4729-bf30-782c8c6ed9fe admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.045492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.046287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.046628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.047266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] PATH_INFO: `/v3/projects/cfe2c527411d4eceb6036eeb890693d6/users/9157d6eff3724042bdd7d57e74a73c92/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.047833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.049735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:48.080242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d6d18-54ba-41d2-85e8-802e224cc191 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.102835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.103489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.103763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.104066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] PATH_INFO: `/v3/projects/ef24d9e55c6e4cd1be40a094c6aa6422` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.106675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef24d9e55c6e4cd1be40a094c6aa6422)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.109793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe2c527411d4eceb6036eeb890693d6, user_id=9157d6eff3724042bdd7d57e74a73c92, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.112282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.118889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.254904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 860/1697] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:48 2022] PUT /identity/v3/projects/cfe2c527411d4eceb6036eeb890693d6/users/9157d6eff3724042bdd7d57e74a73c92/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:20:48.260186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.434365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef24d9e55c6e4cd1be40a094c6aa6422', 'request_id': 'req-0e5414ef-c16a-4316-8b00-dce5fad2eba5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:48.479525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef24d9e55c6e4cd1be40a094c6aa6422', 'request_id': 'req-0e5414ef-c16a-4316-8b00-dce5fad2eba5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:48.509979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:48.536843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72718494-ba33-4f2a-9b83-48afc657e9bf admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.609437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef24d9e55c6e4cd1be40a094c6aa6422', 'request_id': 'req-0e5414ef-c16a-4316-8b00-dce5fad2eba5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:48.648874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] The token cache is being invalidate because project ef24d9e55c6e4cd1be40a094c6aa6422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:48.649201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:48.654344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:48.658946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.658946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.659192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.659452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.661181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 838/1698] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:48 2022] DELETE /identity/v3/projects/ef24d9e55c6e4cd1be40a094c6aa6422 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:48.661548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.663614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.667480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.670242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:48.675718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:48.707195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 861/1699] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:48.708206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.721293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:48.728683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5414ef-c16a-4316-8b00-dce5fad2eba5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.748107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.748107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.748107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.748107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] PATH_INFO: `/v3/users/c5cd8ccb1d844fdcb41fd89dbe646b80` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.756703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cd8ccb1d844fdcb41fd89dbe646b80)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.771847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.813095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc50d4f-0bad-477f-88e3-b6fd7994fd39 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.848280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.852880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.853158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.853434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] PATH_INFO: `/v3/projects/cfe2c527411d4eceb6036eeb890693d6/users/9157d6eff3724042bdd7d57e74a73c92/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.867103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe2c527411d4eceb6036eeb890693d6, user_id=9157d6eff3724042bdd7d57e74a73c92, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.870122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.896679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 862/1700] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:48 2022] PUT /identity/v3/projects/cfe2c527411d4eceb6036eeb890693d6/users/9157d6eff3724042bdd7d57e74a73c92/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:48.896679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:48.926486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:49.109135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731fc1a-e11d-446d-b5ab-f417d8a25cd1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:49.138482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:49.139519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:49.139802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:49.141561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:49.146807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:49.161683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:49.164659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291712c-8d41-4464-a3f6-643e941d923b service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:49.177827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 863/1701] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:49.179608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:49.182059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:49.187733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:49.221985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cd8ccb1d844fdcb41fd89dbe646b80'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:49.264091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] MFA Rules not processed for user `9157d6eff3724042bdd7d57e74a73c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:49.264091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:49.309748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cd8ccb1d844fdcb41fd89dbe646b80'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:49.311351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cd8ccb1d844fdcb41fd89dbe646b80'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:49.322934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cd8ccb1d844fdcb41fd89dbe646b80'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:49.403183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cd8ccb1d844fdcb41fd89dbe646b80'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:49.415284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 839/1702] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:48 2022] DELETE /identity/v3/users/c5cd8ccb1d844fdcb41fd89dbe646b80 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:49.498845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 864/1703] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:49.739960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:49.742067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:49.897114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace9dfd-a759-4d79-bc81-c31d32b9e1b7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:49.913697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:49.915280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:49.921421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:49.923677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:49.926332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:49.929031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:49.933110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:50.116041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66385f-a30e-4e4a-a565-c8df6adefd3e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:50.149057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:50.154031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b93c72-8fe2-4eef-910b-362b05720a5b service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:50.203143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 840/1704] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:50.260191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:50.260637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:50.261193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:50.261669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] PATH_INFO: `/v3/projects/b634224115b042b68696f82f0ab99ff9` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:50.263353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b634224115b042b68696f82f0ab99ff9)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:50.264798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: WARNING py.warnings [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:20:50.264798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: warnings.warn(msg) Aug 29 19:20:50.264798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: Aug 29 19:20:50.266338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:50.528515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b634224115b042b68696f82f0ab99ff9', 'request_id': 'req-cd66def7-b907-414c-83ed-a404070704d0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:50.570379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b634224115b042b68696f82f0ab99ff9', 'request_id': 'req-cd66def7-b907-414c-83ed-a404070704d0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:50.611355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b634224115b042b68696f82f0ab99ff9', 'request_id': 'req-cd66def7-b907-414c-83ed-a404070704d0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:50.659647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd096d6-5ad2-496b-83a4-3c3ec3d07b32 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:50.660020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd096d6-5ad2-496b-83a4-3c3ec3d07b32 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:50.660376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd096d6-5ad2-496b-83a4-3c3ec3d07b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:50.725911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1bd096d6-5ad2-496b-83a4-3c3ec3d07b32 None None] MFA Rules not processed for user `bd993c0493d447cb9b02e424a28e24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:50.727903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1bd096d6-5ad2-496b-83a4-3c3ec3d07b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:50.739544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:50.823947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] The token cache is being invalidate because project b634224115b042b68696f82f0ab99ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:50.827407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:50.827407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cd66def7-b907-414c-83ed-a404070704d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:50.828565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 865/1705] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:49 2022] DELETE /identity/v3/projects/b634224115b042b68696f82f0ab99ff9 => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:50.832288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d743cc8-fde9-4661-aa4f-9884878b0c38 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:50.832937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d743cc8-fde9-4661-aa4f-9884878b0c38 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:50.837604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d743cc8-fde9-4661-aa4f-9884878b0c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:50.887672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 841/1706] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:50.891396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:50.893633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:50.894220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:50.917374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1d743cc8-fde9-4661-aa4f-9884878b0c38 None None] MFA Rules not processed for user `74bfde91bf944824bdae8207641a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:50.921633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1d743cc8-fde9-4661-aa4f-9884878b0c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:50.957828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:50.967293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.030907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:51.152731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 866/1707] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:51.152731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915815a6-6db6-4cd7-8569-6bc40f6b4d39 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.152731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915815a6-6db6-4cd7-8569-6bc40f6b4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.153737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915815a6-6db6-4cd7-8569-6bc40f6b4d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.172581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 842/1708] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:51.173671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.177437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.185534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-915815a6-6db6-4cd7-8569-6bc40f6b4d39 None None] MFA Rules not processed for user `bd993c0493d447cb9b02e424a28e24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:51.187221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-915815a6-6db6-4cd7-8569-6bc40f6b4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.217640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 867/1709] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:51.220537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.221053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.221555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.276428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:51.278581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.293892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 868/1710] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:51.296380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.297972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.304563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366c3a4-0518-4137-bcd1-ca4f19e5a9ec None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.310866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.312036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.312480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.313544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.321194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317565-2ad8-4bec-b644-458ceeb8cb17 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.323563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.325583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.328610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.329601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.330005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.330482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.339811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 843/1711] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:51.340936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.343653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.346144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.353546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297842b-0e46-4de2-8113-07b2327ff9aa service nova] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.370757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 869/1712] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:51.374456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.376131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.377353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.377838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.382287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:51.383108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.385092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.401586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 844/1713] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:51.406645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.426245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2c06a-3d29-4dd7-b32a-1267314c9b9a service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.446372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.449044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.449449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.450532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.453011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.462438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.685551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 870/1714] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:51 2022] POST /identity/v3/projects => generated 365 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:51.689565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.703750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d8796-c7ee-4f08-b0c5-0b803233df4a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.860000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.860928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.861383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.861798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.862592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.865460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.904520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:51.906576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:51.951095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b789-c59d-4b16-92ef-b87c983c461a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:51.982259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:51.983217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:51.983937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:51.984364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] PATH_INFO: `/v3/users/0fc03a7273624d98bdfb4b7846c3a78f` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:51.987457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc03a7273624d98bdfb4b7846c3a78f)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:51.991437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:51.992895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 845/1715] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:51 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:51.997512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:52.019380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46b80-111c-415a-8720-41250b37c307 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:52.040355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:52.040355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.040355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.040355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.040692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:52.044477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:52.062420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 846/1716] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:52.071214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:52.099401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a767ef-9162-4928-9caf-721d171829ab admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:52.116345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:52.117171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.117457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.117725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.118727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:52.120555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:52.141033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 847/1717] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:52.148999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:52.180817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f778-fe13-49f5-8baf-3e9e3c22219b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:52.195156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:52.196230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.196856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.197317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] PATH_INFO: `/v3/projects/ba464c95205f423bb3b5f55a5e4dfd63/users/e6b3dde2253240c18d33fba23b8329ed/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.244393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba464c95205f423bb3b5f55a5e4dfd63, user_id=e6b3dde2253240c18d33fba23b8329ed, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:52.247505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345e541-008c-4f35-a752-3aaff67529c6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:52.297163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc03a7273624d98bdfb4b7846c3a78f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:52.299634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc03a7273624d98bdfb4b7846c3a78f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:52.326797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc03a7273624d98bdfb4b7846c3a78f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:52.336654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc03a7273624d98bdfb4b7846c3a78f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:52.413065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 848/1718] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:52 2022] PUT /identity/v3/projects/ba464c95205f423bb3b5f55a5e4dfd63/users/e6b3dde2253240c18d33fba23b8329ed/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:52.414492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.424215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.424496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.460472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-117a77a6-17d2-41e7-b855-276d5d655c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc03a7273624d98bdfb4b7846c3a78f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:52.479405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] MFA Rules not processed for user `e6b3dde2253240c18d33fba23b8329ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:52.480668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:52.489143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 871/1719] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:51 2022] DELETE /identity/v3/users/0fc03a7273624d98bdfb4b7846c3a78f => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:20:52.492853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.493793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.500729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.557750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:52.559091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:52.679640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 849/1720] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:52.685561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:52.690111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:52.763103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 872/1721] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:52.772426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:52.774143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:52.881391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edec73-7b6e-4d1d-b895-a49bc28de3ac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:52.883788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1acaa6-bb80-4542-8ab8-cf89e4c799b8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:52.892317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:52.893030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.893381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.893706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:52.893845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.895871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:52.896444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:52.896721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:52.897581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:52.900082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eebbd0-c16b-4d28-bed6-046f576c3ad1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:52.914813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:52.916496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:52.934623 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 873/1722] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:53.065332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 850/1723] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:52 2022] POST /identity/v3/projects => generated 365 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:53.072749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:53.125559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa9cc5-571f-43ee-826a-2937410bad62 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:53.202309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:53.202593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:53.202877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:53.289159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] MFA Rules not processed for user `bd993c0493d447cb9b02e424a28e24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:53.291410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:53.404687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:53.406187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:53.406721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:53.407104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:53.407968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:53.411143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:53.460786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 851/1724] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:20:53.562098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 874/1725] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:53 2022] POST /identity/v3/users => generated 289 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:53.566002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:53.596673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545708-3684-42bb-8157-f2b320c1c99d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:53.607002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:53.637190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:53.637718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:53.638985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:53.651789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:53.663891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:53.666525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:53.668802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:53.726849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b90762-1479-4603-a348-0346b7094733 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:53.752779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 852/1726] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:53.759394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:53.760507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:53.761166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:53.761458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:53.761794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] PATH_INFO: `/v3/users/8784201e56be42529494c91dc566c898` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:53.772827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8784201e56be42529494c91dc566c898)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:53.782400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:53.805411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864dfe-2f17-4a48-a151-646896dbabce admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:53.824536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:53.825010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:53.826065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:53.826065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] PATH_INFO: `/v3/projects/ef57419a1206491cad5d733a9d818540/users/c6ed29506969418aac8d3f7ba690fd51/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:53.862822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef57419a1206491cad5d733a9d818540, user_id=c6ed29506969418aac8d3f7ba690fd51, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:53.866823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4b02-7b9b-4b2a-ba32-c9d3c1127317 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:54.053274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 853/1727] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:53 2022] PUT /identity/v3/projects/ef57419a1206491cad5d733a9d818540/users/c6ed29506969418aac8d3f7ba690fd51/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:54.053274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e1e04-8c4b-4acc-9601-6bf6b2a7c3e3 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.053557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e1e04-8c4b-4acc-9601-6bf6b2a7c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.053834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e1e04-8c4b-4acc-9601-6bf6b2a7c3e3 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.060474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 854/1728] 10.222.0.62 () {54 vars in 912 bytes} [Mon Aug 29 19:20:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:20:54.061514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e1e04-8c4b-4acc-9601-6bf6b2a7c3e3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:54.212110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8784201e56be42529494c91dc566c898'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.215274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8784201e56be42529494c91dc566c898'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.219124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e1e04-8c4b-4acc-9601-6bf6b2a7c3e3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:54.229918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:54.230724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.231184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.231357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.232341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:54.234455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8784201e56be42529494c91dc566c898'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.234822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca676be4-a1ac-44e0-ad8e-ccdb1c9d655d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:54.250139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 855/1729] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:54.250139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.250139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.250139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.254546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8784201e56be42529494c91dc566c898'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.293762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:54.297640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:54.365357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 856/1730] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:20:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:54.366156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:54.375227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:54.378609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8784201e56be42529494c91dc566c898'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.385469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 875/1731] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:53 2022] DELETE /identity/v3/users/8784201e56be42529494c91dc566c898 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:20:54.386297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:54.388826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:54.412499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf821e55-565e-472c-a36b-8be6a7f2fb33 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:54.601637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d013-f84a-4058-9f01-afa51a1f7ae3 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:54.602137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:54.602137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.602137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.602425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] PATH_INFO: `/v3/projects/29fc03040cd14f6cb4246bef42ae2166` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.604629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fc03040cd14f6cb4246bef42ae2166)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:54.606876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:54.613802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:54.614501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.614773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.615046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] PATH_INFO: `/v3/projects/ef57419a1206491cad5d733a9d818540/users/c6ed29506969418aac8d3f7ba690fd51/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.628165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef57419a1206491cad5d733a9d818540, user_id=c6ed29506969418aac8d3f7ba690fd51, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:54.633469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75612b-9f44-4015-b226-c7b3045bb581 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:54.658270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 876/1732] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:54 2022] PUT /identity/v3/projects/ef57419a1206491cad5d733a9d818540/users/c6ed29506969418aac8d3f7ba690fd51/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:54.659770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:54.660706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:54.661383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:54.732852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:54.741096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:54.858440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 877/1733] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:54.868434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:54.873135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:54.875787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fc03040cd14f6cb4246bef42ae2166', 'request_id': 'req-111176f5-5b0a-4119-bb39-524913e1b216'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.889402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fc03040cd14f6cb4246bef42ae2166', 'request_id': 'req-111176f5-5b0a-4119-bb39-524913e1b216'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:54.980008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134294a-d4e3-4fcb-9c8c-8259c0c29457 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.049958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.051057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.051304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.051670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.053803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.064560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.066916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa01d4-e47f-49c6-9d36-6a3932e953c2 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.076348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 878/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:55.094224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.113213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.113213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.141036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] MFA Rules not processed for user `c6ed29506969418aac8d3f7ba690fd51`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:55.142432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.149921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fc03040cd14f6cb4246bef42ae2166', 'request_id': 'req-111176f5-5b0a-4119-bb39-524913e1b216'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.172327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] The token cache is being invalidate because project 29fc03040cd14f6cb4246bef42ae2166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:55.172950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.174648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.180899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 857/1735] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:54 2022] DELETE /identity/v3/projects/29fc03040cd14f6cb4246bef42ae2166 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:55.183069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:55.185958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.207885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:55.219378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111176f5-5b0a-4119-bb39-524913e1b216 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.231879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.235949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.236455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.237222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.242297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.354075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 879/1736] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:55.354075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:55.354075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.394637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.394637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9622e97-7102-44ae-b10b-7e2904fa4f24 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.417464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea979de3-ed88-4abd-b4bd-893a39a9499b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.422619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.425323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.425323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.425323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] PATH_INFO: `/v3/users/00c1651cd24b4bb2ac09945a16aba093` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.426239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 858/1737] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:55.426368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c1651cd24b4bb2ac09945a16aba093)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.427107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.427597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.428061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.428159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.455432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:55.457981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.472297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 859/1738] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:55.474136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:55.475877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.546551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f1388-9d29-4770-b57b-f9bd8f9754b6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.551931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.552734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.553575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.554180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.576738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.578475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13a479-f3ef-4116-b218-fc80e7742e0f service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.592983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 860/1739] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:55.595205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.595647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.596046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.625601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:55.627091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:55.660549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 861/1740] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:55.664132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:55.679454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9361c88-b888-47dd-a4de-178d729b9fce None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.696499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.697139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.697418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.698367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.699049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:20:55.699420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.702024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.719528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 862/1741] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:20:55.729254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:55.742093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134758c-d49f-496b-81ac-1931f14f1473 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:55.757750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c1651cd24b4bb2ac09945a16aba093'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.759387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c1651cd24b4bb2ac09945a16aba093'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.763828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:55.766852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:55.766852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:55.766852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:55.766852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:55.773333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:55.783909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c1651cd24b4bb2ac09945a16aba093'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.793144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c1651cd24b4bb2ac09945a16aba093'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:55.951454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:56.059267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c1651cd24b4bb2ac09945a16aba093'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:56.074711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 863/1742] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:55 2022] POST /identity/v3/projects => generated 385 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:56.081833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.084543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.092042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 880/1743] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:55 2022] DELETE /identity/v3/users/00c1651cd24b4bb2ac09945a16aba093 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:56.093695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.098301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.196023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4a734-60a6-43ac-8082-ba91560297a1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.201916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.202653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.202941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.203212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.203846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.205956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.212110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211d946-ec4e-495c-bc26-e06bc130513c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.267714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.268430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.268834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.269108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.270966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.335413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.336998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a967de-2d67-430b-8487-714d69a55055 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.346851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 864/1744] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:56.348507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.348799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.349074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.370139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:56.371436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.382443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 865/1745] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:56.393390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.396747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.431941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 881/1746] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:56 2022] POST /identity/v3/users => generated 293 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:56.433345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.457383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd8a4-54f0-454a-894a-5248239da0c4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.462506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.463529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.464494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.465155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.466104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.468364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.480816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 882/1747] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:20:56.486247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.496703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd71a-fe62-408b-b6f8-c8da0912d7ce admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.497337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a519b4e-12a6-46da-a4aa-031320040004 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.502019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.502200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.502734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.502922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.503203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.503345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.503478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.504772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.504879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.506369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.516436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.518088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.520826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 883/1748] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:56.536967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 866/1749] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:56.539003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.563125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d38bb8-683d-466a-b16d-88b16256c161 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.579766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.580543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.580825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.585319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] PATH_INFO: `/v3/projects/2ac7903f83c4416798792674b82d6585/users/5e8b78af958c4a019380cc24701a2006/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.621859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac7903f83c4416798792674b82d6585, user_id=5e8b78af958c4a019380cc24701a2006, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.628365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e5d9c-ac9d-4215-8886-b9beeaef9ac4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.637737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:56.665859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:56.700436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c2b0-a34c-486f-90b0-a6ccb0c206f4 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:56.718602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:56.724451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.724859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.725150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] PATH_INFO: `/v3/projects/11820555aa474e98954000a1238ece79` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.726670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11820555aa474e98954000a1238ece79)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:56.728746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:56.854244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 884/1750] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:56 2022] PUT /identity/v3/projects/2ac7903f83c4416798792674b82d6585/users/5e8b78af958c4a019380cc24701a2006/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:56.867080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:56.867575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:56.868150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:56.909436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11820555aa474e98954000a1238ece79', 'request_id': 'req-d45d8a2c-2f15-4156-8ca1-f6853a306661'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:56.922841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11820555aa474e98954000a1238ece79', 'request_id': 'req-d45d8a2c-2f15-4156-8ca1-f6853a306661'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:56.922973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] MFA Rules not processed for user `5e8b78af958c4a019380cc24701a2006`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:56.924361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:57.015024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11820555aa474e98954000a1238ece79', 'request_id': 'req-d45d8a2c-2f15-4156-8ca1-f6853a306661'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.036490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] The token cache is being invalidate because project 11820555aa474e98954000a1238ece79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:57.037233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.040538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d45d8a2c-2f15-4156-8ca1-f6853a306661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.046774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 867/1751] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:56 2022] DELETE /identity/v3/projects/11820555aa474e98954000a1238ece79 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:57.046774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.046774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.046774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.055044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 885/1752] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:57.056173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.094520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] MFA Rules not processed for user `74bfde91bf944824bdae8207641a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:57.097122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:57.191832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a738ae-048d-44e5-8d6d-145ed5f6e7ab None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.200348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.201078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.201366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.201642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] PATH_INFO: `/v3/users/558042483ae74198bce7e1306ceb4d9c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.202985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558042483ae74198bce7e1306ceb4d9c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.207172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:57.258086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 868/1753] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:57.259505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.269122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:57.308628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85e447-b800-4d7b-9810-be27ecbd1a67 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.320983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.321879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.322300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.322717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.323524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.328373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:57.532948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 869/1754] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:20:57 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:57.537605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.567477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558042483ae74198bce7e1306ceb4d9c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.568532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558042483ae74198bce7e1306ceb4d9c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.599054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558042483ae74198bce7e1306ceb4d9c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.614494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558042483ae74198bce7e1306ceb4d9c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.671494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a5b6-b27f-44e3-8ebc-9159cc98ef91 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.677618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.678261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.689015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.689015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.689015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.689015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:57.725776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558042483ae74198bce7e1306ceb4d9c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:57.736875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 886/1755] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:57 2022] DELETE /identity/v3/users/558042483ae74198bce7e1306ceb4d9c => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:57.738653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.740306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:57.828020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 870/1756] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:20:57 2022] POST /identity/v3/users => generated 299 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:20:57.832798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.834485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:57.852415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff7e-67ea-42af-9eb9-bc6683c1d9c9 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.855205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c3dcd-fd77-470b-9a80-ab97d712f3c3 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.858713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.861029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.861029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.861029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.861562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.862704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.863764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.864649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:57.864809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.865094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] PATH_INFO: `/v3/projects/654eea424bd24f2bb447382c435f2c1d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.874278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654eea424bd24f2bb447382c435f2c1d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.877288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:57.885679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 871/1757] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:57.893979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:57.907141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24933c2-bf87-4396-b367-f76501285697 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:57.916847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:57.918133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:57.919132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:57.919808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] PATH_INFO: `/v3/projects/6980dd3e7bb947d990318ddfa371397c/users/b2c6f425dde1472fa8800b72a4a865da/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:57.951768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6980dd3e7bb947d990318ddfa371397c, user_id=b2c6f425dde1472fa8800b72a4a865da, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:57.955043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:58.091032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654eea424bd24f2bb447382c435f2c1d', 'request_id': 'req-8c7d8dd6-a207-4b59-9882-71ec396b96b4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.106316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 872/1758] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:57 2022] PUT /identity/v3/projects/6980dd3e7bb947d990318ddfa371397c/users/b2c6f425dde1472fa8800b72a4a865da/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:58.110704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654eea424bd24f2bb447382c435f2c1d', 'request_id': 'req-8c7d8dd6-a207-4b59-9882-71ec396b96b4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.115756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:58.122773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654eea424bd24f2bb447382c435f2c1d', 'request_id': 'req-8c7d8dd6-a207-4b59-9882-71ec396b96b4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.265434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e8a39-7196-4460-a88a-0f26908bacc8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:58.296262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] The token cache is being invalidate because project 654eea424bd24f2bb447382c435f2c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:58.296715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.299026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8c7d8dd6-a207-4b59-9882-71ec396b96b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.301944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 887/1759] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:57 2022] DELETE /identity/v3/projects/654eea424bd24f2bb447382c435f2c1d => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:58.303361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:58.305348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:58.305988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:58.407972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:58.409812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:58.412326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:58.412607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:58.413452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:58.415473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:58.426400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:58.428355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:58.432236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 873/1760] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:20:58.433535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:58.437811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:58.459554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc802b-639b-410b-8249-8d30e2234de2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:58.472339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:58.473163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:58.473549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:58.473899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] PATH_INFO: `/v3/users/5a3b6976eb684fbf8c28f60fed2b1464` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:58.477088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3b6976eb684fbf8c28f60fed2b1464)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:58.480318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:58.530470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 888/1761] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:58.530470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:58.530470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:58.618506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93a556-735d-4f14-93b3-86929774b0bf None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:58.623218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:58.624010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:58.624401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:58.624899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:58.626895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:58.748704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:58.752774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:58.814577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 889/1762] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:58.815499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:58.819287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:58.878337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3b6976eb684fbf8c28f60fed2b1464'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:58.881441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632bd76-eba4-4479-bb9f-2cc453f68c8e service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:58.891356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:58.892063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:58.892345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:58.892621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] PATH_INFO: `/v3/projects/6980dd3e7bb947d990318ddfa371397c/users/b2c6f425dde1472fa8800b72a4a865da/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:58.896683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6980dd3e7bb947d990318ddfa371397c, user_id=b2c6f425dde1472fa8800b72a4a865da, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:58.899455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:58.929588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 890/1763] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:20:58 2022] PUT /identity/v3/projects/6980dd3e7bb947d990318ddfa371397c/users/b2c6f425dde1472fa8800b72a4a865da/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:58.930629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:58.938331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.046474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3b6976eb684fbf8c28f60fed2b1464'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.046474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3b6976eb684fbf8c28f60fed2b1464'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.056672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3b6976eb684fbf8c28f60fed2b1464'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.077011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1f1be6-1386-4587-ae16-59ab151856e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3b6976eb684fbf8c28f60fed2b1464'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.092326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 874/1764] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:20:58 2022] DELETE /identity/v3/users/5a3b6976eb684fbf8c28f60fed2b1464 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:59.110271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:59.110720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:59.111171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:59.202194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] MFA Rules not processed for user `b2c6f425dde1472fa8800b72a4a865da`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:59.203604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.214697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfaf7-eb3f-41da-bdf2-78cbe3b27fa5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:59.299048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:59.317672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 875/1765] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:20:59.319083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:59.320624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.330061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:59.330722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:59.331131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:59.333051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] PATH_INFO: `/v3/projects/13186aa95168469c90ec357870d80687` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:59.334326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13186aa95168469c90ec357870d80687)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:59.336381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:59.455079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae1415-6b58-4c48-981c-b387f0fd2e46 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:59.464561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:59.465438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:59.465960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:59.466455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:59.468765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.596148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13186aa95168469c90ec357870d80687', 'request_id': 'req-2e556a1e-d94d-45b6-9e79-7c11fd797090'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.622645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13186aa95168469c90ec357870d80687', 'request_id': 'req-2e556a1e-d94d-45b6-9e79-7c11fd797090'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.631398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13186aa95168469c90ec357870d80687', 'request_id': 'req-2e556a1e-d94d-45b6-9e79-7c11fd797090'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.645206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:59.650655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:59.667524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 876/1766] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:20:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:20:59.669320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:59.672532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.807823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] The token cache is being invalidate because project 13186aa95168469c90ec357870d80687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:20:59.808538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.812291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2e556a1e-d94d-45b6-9e79-7c11fd797090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:20:59.817002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 891/1767] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:20:58 2022] DELETE /identity/v3/projects/13186aa95168469c90ec357870d80687 => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:20:59.822491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3ea9-2dfe-4da7-bbdb-ac217bf08ee7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:59.823447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3ea9-2dfe-4da7-bbdb-ac217bf08ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:59.823447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3ea9-2dfe-4da7-bbdb-ac217bf08ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:59.909477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e47c3ea9-2dfe-4da7-bbdb-ac217bf08ee7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:20:59.914282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e47c3ea9-2dfe-4da7-bbdb-ac217bf08ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:59.934652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5605ea-d0d5-4df8-8581-4ae2469c33af service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:00.122752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:00.123013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 892/1768] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:00.125435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.129507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.129507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.129507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.130682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.132607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.143165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:00.200200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:00.203230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:00.212978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 893/1769] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:00.222804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:00.234755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc71119-83df-48d4-aa63-f4d101be665e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:00.245244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:00.249244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:00.270315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 894/1770] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:00.271601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:00.285349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678021e1-435a-4cae-b61b-e95d013ab259 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:00.291776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:00.292581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.292965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.293322 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.294349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:00.298372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:00.385874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:00.393921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf6a11-0e33-44e4-9f7f-dd3978ae75d2 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:00.407313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 877/1771] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:00.435414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.435997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.438772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.483119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 895/1772] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:00 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:00.488084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:00.543211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:00.552717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:00.715272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 878/1773] 10.222.0.76 () {58 vars in 1015 bytes} [Mon Aug 29 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:00.716412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:00.718694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:00.886145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2faa48-e529-4f3c-8598-d8c69148ae20 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:00.891782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:00.893169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.893646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.893966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] PATH_INFO: `/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.896238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3a9d4b88954972a23c0e8d2f306c38)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:00.898308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:00.935541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1330a1-c3e9-4301-a498-4552b4535f4f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:00.949631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:00.950348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:00.950826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:00.951177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:00.951957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:00.953900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:01.216666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3a9d4b88954972a23c0e8d2f306c38', 'request_id': 'req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:01.223384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3a9d4b88954972a23c0e8d2f306c38', 'request_id': 'req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:01.251017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 896/1774] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:00 2022] POST /identity/v3/users => generated 291 bytes in 763 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:01.251918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:01.263925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:01.288847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09505a-3c13-405e-8b93-4b7368374840 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:01.353075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:01.568818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 897/1775] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:01.569649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:01.569649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:01.569649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9fcc5-f132-4597-a86a-ef220152ec4f service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:01.570300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:01.570300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:01.570300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:01.570300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] PATH_INFO: `/v3/users/5e8b78af958c4a019380cc24701a2006` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:01.571001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8b78af958c4a019380cc24701a2006)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:01.571001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:01.880620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3a9d4b88954972a23c0e8d2f306c38', 'request_id': 'req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:01.896011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] The token cache is being invalidate because project ba3a9d4b88954972a23c0e8d2f306c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:01.896348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:01.897030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:01.898564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 879/1776] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:00 2022] DELETE /identity/v3/projects/ba3a9d4b88954972a23c0e8d2f306c38 => generated 0 bytes in 1183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:01.899436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:01.901124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:01.914928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419465e0-4a8d-49f0-93f6-f35dc9fd4d54 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:01.927227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:01.927921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:01.928171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:01.928446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:01.929096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:01.930749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e775-c6d0-4af0-9300-4d993801739f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:01.942891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 880/1777] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:01.944173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:01.944452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:01.944749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:01.965049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:01.966265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:01.978181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 881/1778] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:01.979028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:01.989023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db74dd-3dfb-4421-bb50-ad2ab8edf5b5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:01.993693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:01.994301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:01.994581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:01.994874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:01.995506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:01.997137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f21ce-604b-4025-acc1-8c0a4581450a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.008425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 882/1779] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:02.009891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.010229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.010662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.031217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:02.032462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.040550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 883/1780] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:02.041408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.053624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825e6a-cabc-496d-851b-c7a557736b19 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.058086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.058740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.059012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.059282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] PATH_INFO: `/v3/projects/7d88ef34b22242b3afb6a277772cc88c/users/c982237cf2c5473ca61ca26c1108f379/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.075118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d88ef34b22242b3afb6a277772cc88c, user_id=c982237cf2c5473ca61ca26c1108f379, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.078007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.132426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8b78af958c4a019380cc24701a2006'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:02.133147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8b78af958c4a019380cc24701a2006'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:02.150397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8b78af958c4a019380cc24701a2006'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:02.161475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8b78af958c4a019380cc24701a2006'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:02.283033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 884/1781] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:02 2022] PUT /identity/v3/projects/7d88ef34b22242b3afb6a277772cc88c/users/c982237cf2c5473ca61ca26c1108f379/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:02.284174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.346087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ba34fd0-d494-449a-b3fa-50ab55a66326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8b78af958c4a019380cc24701a2006'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:02.362477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 898/1782] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:01 2022] DELETE /identity/v3/users/5e8b78af958c4a019380cc24701a2006 => generated 0 bytes in 989 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:02.366646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.367151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.367591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.436060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40633-7846-48cf-ac8e-0b7dd7c458ef admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.440730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] MFA Rules not processed for user `c982237cf2c5473ca61ca26c1108f379`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:02.444428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.526917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.528255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.528703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.529201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.530067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:02.530518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.538424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffc5b4-5570-4566-850d-f0a18e79f50d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.549286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 885/1783] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:02.550864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.551333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.551811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.567806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 899/1784] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:02.569803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.571402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.595836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ff05-baa0-41f9-ac39-a047d5fc8930 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.596060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:02.597799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.601874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.602799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.603081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.603360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.604106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.606452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.611843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 886/1785] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:02.613020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.614747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.627655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d7288-1667-4ff6-bb45-15aae2c62f6b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.639008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.641278 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.641566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.641836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.642480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.644808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990858c2-af98-46fe-bca7-5bf63966fcf5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.665972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:02.728324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 900/1786] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:02.729046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.735427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:02.793625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 887/1787] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:02.793625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf6b3-8560-4e0f-8155-c6ce0678bd2a None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.793625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf6b3-8560-4e0f-8155-c6ce0678bd2a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.793625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf6b3-8560-4e0f-8155-c6ce0678bd2a None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.807972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 888/1788] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:21:02.807972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bf6b3-8560-4e0f-8155-c6ce0678bd2a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.827577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3187-d381-44bf-87be-c10b8d5a8f24 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.890194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.891125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.891403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.891701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.897894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bf6b3-8560-4e0f-8155-c6ce0678bd2a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.903438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.903554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.903554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.903724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.903798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.903798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.904315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.906241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44fd93-e958-4286-8eb6-83c0567849f6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:02.913554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 901/1789] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:02.914420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:02.925121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650d8b7-b3a4-4b74-b522-7e16031ffa7e service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:02.930015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:02.930639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:02.930912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:02.931178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:02.931810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:02.933767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.059137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 889/1790] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/users => generated 284 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:03.059137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.059137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.059137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.107600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:03.111474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 902/1791] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:02 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:03.112201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:03.113237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.137020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4b97c-802d-4e87-8c35-6dc7801e2e2f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.149707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.150522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.151127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.151556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.152933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.155453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.186547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 903/1792] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:03.192584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.205066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f88b5-3c52-467c-8926-ae8830a63cf7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.210917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.212551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.212551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.212551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.212962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.214229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.231906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 904/1793] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:03.231906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.245556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 890/1794] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:03.246625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.262463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e3340-63d1-45ad-bcf7-68297190068d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.263926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b17e-5556-4acd-aa64-0aad5472bf73 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.265874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.266630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.266970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.267376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] PATH_INFO: `/v3/projects/49bbf675d88d4377b8c21d5384b789f7/users/916a7b57790e43beb7b299a29e725609/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.272652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.273344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.273684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.273969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.274614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.276350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.285276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bbf675d88d4377b8c21d5384b789f7, user_id=916a7b57790e43beb7b299a29e725609, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.286842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 905/1795] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:03.287662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.288112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.289319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:03.305385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648762c-eecd-4ab7-a436-584bdc2b1e8e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.311525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.312168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.312443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.312711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.322219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.323837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.335025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 906/1796] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:03.335835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.352051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0658d6-7bbd-4407-a04f-1461c09884a4 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.362213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.367422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.367422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.367422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] PATH_INFO: `/v3/projects/2c7f9b4bb6c34f91b03415295ecb6038/users/f13e9d788e2244ea9de44c4b4b2eb447/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.384443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7f9b4bb6c34f91b03415295ecb6038, user_id=f13e9d788e2244ea9de44c4b4b2eb447, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.387406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffa326-2ceb-4477-a6e8-d22fa013cbd1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.443486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 891/1797] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:03 2022] PUT /identity/v3/projects/49bbf675d88d4377b8c21d5384b789f7/users/916a7b57790e43beb7b299a29e725609/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:03.443486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.502172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3e89-c55c-4b3b-996b-a4d0a60a0cce admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.507720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.508385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.508662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.508929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.509600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.511189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.521784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 892/1798] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:03.525867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.535321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cb53c-2c00-4b59-a6a1-7b35f8d300ea admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.540569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.541230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.541507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.541775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] PATH_INFO: `/v3/projects/49bbf675d88d4377b8c21d5384b789f7/users/916a7b57790e43beb7b299a29e725609/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.543286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 907/1799] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:03 2022] PUT /identity/v3/projects/2c7f9b4bb6c34f91b03415295ecb6038/users/f13e9d788e2244ea9de44c4b4b2eb447/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:03.543660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bbf675d88d4377b8c21d5384b789f7, user_id=916a7b57790e43beb7b299a29e725609, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.546357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9abe3b-1807-43f7-a7f3-77db79315c5b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.557225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 893/1800] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:03 2022] PUT /identity/v3/projects/49bbf675d88d4377b8c21d5384b789f7/users/916a7b57790e43beb7b299a29e725609/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:03.561771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.562118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.562400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.567615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.567924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.568197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.584410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:03.593348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] MFA Rules not processed for user `f13e9d788e2244ea9de44c4b4b2eb447`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:03.594589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:03.603070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] MFA Rules not processed for user `916a7b57790e43beb7b299a29e725609`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:03.604855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:03.668717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 908/1801] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:03.669793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.671060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:03.694824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 894/1802] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:03.695930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:03.772767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f7eac-e574-4b01-aca6-f16f3f2e7397 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.778243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.786564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.786564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.786564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] PATH_INFO: `/v3/projects/2ac7903f83c4416798792674b82d6585` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.786564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac7903f83c4416798792674b82d6585)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.786564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:03.798349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a95023-5c46-4221-954c-8dfe8a23e71a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:03.814634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:03.815538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:03.818834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:03.819296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:03.820181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:03.822873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.027492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 895/1803] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:04.028465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.035645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:04.058996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac7903f83c4416798792674b82d6585', 'request_id': 'req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.058996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac7903f83c4416798792674b82d6585', 'request_id': 'req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.079951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac7903f83c4416798792674b82d6585', 'request_id': 'req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.131510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b79b5-2f3c-4c28-a370-730242a6d52c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.135621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.136361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.136918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.139635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.236335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.238875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.250351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 896/1804] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:04.251139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.286340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] The token cache is being invalidate because project 2ac7903f83c4416798792674b82d6585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:04.286854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.287827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.290882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 909/1805] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:03 2022] DELETE /identity/v3/projects/2ac7903f83c4416798792674b82d6585 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:04.297800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.299672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:04.352543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba92b2a-7de5-42d0-aaf2-4469363c1a57 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.358741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.359963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.360320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.360665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.361374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.364538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.419795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4002d3-adbb-4b94-a3b8-61e2a6efb3a9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.432509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.436917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.437292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.437696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] PATH_INFO: `/v3/users/b2c6f425dde1472fa8800b72a4a865da` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.439717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c6f425dde1472fa8800b72a4a865da)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.442581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.544745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 897/1806] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:04 2022] POST /identity/v3/users => generated 291 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:04.545736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.547511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:04.562556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ada48-9e24-462f-b275-a3e7f2c1aa86 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.569843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.570981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.571256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.571543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.573617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:04.647192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.648821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.660619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 898/1807] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:04.661280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.662693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:04.687763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb00cd-9e8d-48dc-88d8-ca74df7c6dde service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.716419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.716419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.716419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.716419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.716831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.716919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c6f425dde1472fa8800b72a4a865da'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.717520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c6f425dde1472fa8800b72a4a865da'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.720750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.756145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 899/1808] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:04.756374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c6f425dde1472fa8800b72a4a865da'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.757835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.780217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c6f425dde1472fa8800b72a4a865da'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.786559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff25ff-932e-494c-80f1-c5f695a5c6ec admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:04.791451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:04.798638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:04.798638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:04.798638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] PATH_INFO: `/v3/projects/15d272eb9854425b8eb552ba4b245f2a/users/6fbf576dea5446ae828535a313ee13a6/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:04.810613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d272eb9854425b8eb552ba4b245f2a, user_id=6fbf576dea5446ae828535a313ee13a6, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:04.815935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:04.932237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c6f425dde1472fa8800b72a4a865da'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:04.940139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 910/1809] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:04 2022] DELETE /identity/v3/users/b2c6f425dde1472fa8800b72a4a865da => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:04.956029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 900/1810] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:04 2022] PUT /identity/v3/projects/15d272eb9854425b8eb552ba4b245f2a/users/6fbf576dea5446ae828535a313ee13a6/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:04.956358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:04.958140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:05.113259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8f08-6583-4d94-9f7c-86a7b5c1dfe3 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:05.121081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:05.122973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:05.122973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:05.122973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:05.123646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:05.125708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f026ec2-82e4-4292-aab8-3b462201e9a8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:05.145662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 911/1811] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:05.154457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:05.178969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:05.291002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3a915-16b7-4b1d-940e-86dd3196b8d3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:05.297615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:05.298381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:05.298667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:05.299123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] PATH_INFO: `/v3/projects/15d272eb9854425b8eb552ba4b245f2a/users/6fbf576dea5446ae828535a313ee13a6/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:05.306701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d272eb9854425b8eb552ba4b245f2a, user_id=6fbf576dea5446ae828535a313ee13a6, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:05.310365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:05.330399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 901/1812] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:05 2022] PUT /identity/v3/projects/15d272eb9854425b8eb552ba4b245f2a/users/6fbf576dea5446ae828535a313ee13a6/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:05.342044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:05.342695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:05.342695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:05.398102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] MFA Rules not processed for user `6fbf576dea5446ae828535a313ee13a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:05.401306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:05.592655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 912/1813] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:05.608667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:05.610750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:05.725603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbcda3-a113-484b-9f4b-d6eff70d0c1d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:05.732289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:05.732927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:05.733207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:05.733485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:05.746559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:05.748104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03892c61-7877-4542-ae6e-4125cfe40648 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:05.765628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 902/1814] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:05.825799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:05.827807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:05.945229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0800c-8906-4c9a-8062-ed7d65145f80 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:05.953260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:05.953920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:05.954207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:05.954448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] PATH_INFO: `/v3/projects/6980dd3e7bb947d990318ddfa371397c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:05.955607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6980dd3e7bb947d990318ddfa371397c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:05.958268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:06.126110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6980dd3e7bb947d990318ddfa371397c', 'request_id': 'req-20071031-be7c-425f-a6a4-ed22bc5530b1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:06.132534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6980dd3e7bb947d990318ddfa371397c', 'request_id': 'req-20071031-be7c-425f-a6a4-ed22bc5530b1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:06.139621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6980dd3e7bb947d990318ddfa371397c', 'request_id': 'req-20071031-be7c-425f-a6a4-ed22bc5530b1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:06.262747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] The token cache is being invalidate because project 6980dd3e7bb947d990318ddfa371397c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:06.263034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:06.263683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-20071031-be7c-425f-a6a4-ed22bc5530b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:06.265205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 913/1815] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:05 2022] DELETE /identity/v3/projects/6980dd3e7bb947d990318ddfa371397c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:06.271093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:06.272201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:06.272201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:06.292491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:06.293836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:06.388437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 903/1816] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:06.391184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:06.391694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:06.391694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:06.420679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:06.423130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:06.438580 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 914/1817] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:06.443969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:06.455466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea454f-9faa-4360-bef5-1962f1b05008 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:06.461235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:06.462367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:06.462840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:06.463244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:06.464130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:06.464559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:06.467244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:06.485958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 904/1818] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:06.502992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:06.513604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489393e5-6e97-4cad-bd6c-9c96ccc3fc77 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:06.519614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:06.520275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:06.530157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:06.530448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:06.534314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:06.536475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:06.705324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 915/1819] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:06 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:06.710286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:06.728109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5b233-526e-4b49-8527-6b8ae69f0f3e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:06.825118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:06.826558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:06.827349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:06.827838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:06.828778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:06.831467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:06.871092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:06.985662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 905/1820] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:06 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:06.993485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.006006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ec012-5a00-43f0-9276-11a60f9fcacc admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.011020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.011626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.011941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.012382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.013028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.014631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.025224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 916/1821] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:07.029329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.042427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d54bf-8b08-425f-950f-4610800fe427 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.048652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.049458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.049814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.050263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.051594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.053468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac771500-cf6f-4ead-afcc-e7627686a6de admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.071710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 906/1822] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:07.076343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.086593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b64e6-8a41-403d-bc02-4a93fb08d3d9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.093342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.094031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.094306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.094580 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] PATH_INFO: `/v3/projects/ee16b597944f47e39637677fc62efa1e/users/b185f216ff824a179ae5ac0951f7b2b3/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.114627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee16b597944f47e39637677fc62efa1e, user_id=b185f216ff824a179ae5ac0951f7b2b3, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.117341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.270259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 917/1823] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:07 2022] PUT /identity/v3/projects/ee16b597944f47e39637677fc62efa1e/users/b185f216ff824a179ae5ac0951f7b2b3/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:07.282216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.282795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.282795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.326424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] MFA Rules not processed for user `b185f216ff824a179ae5ac0951f7b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:07.329248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:07.409294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 907/1824] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:07.413892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.491059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fc16-2a9c-4e5c-91fd-9ddb6603a0d1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.498914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.499628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.499981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.500260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.500909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.503330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.637425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 918/1825] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:07 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:07.641569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.652820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145e59-ef4c-4b70-bd99-7b90bf4bb69b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.739176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.739797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.740133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.740368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.741015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.748656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.858358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 908/1826] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:07 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:07.870133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.905567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eebe20-81cd-4bfe-98bf-ffae83af7888 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.917903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.919939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.923669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.923669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:07.923669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:07.933086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:07.952905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 919/1827] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:07.958934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:07.975265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823548-07c8-447d-be63-cd514a8fc61e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:07.983173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:07.984394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:07.984704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:07.993618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] PATH_INFO: `/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5/users/9e8d1b657bdd416199d0586c2925272e/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:08.006010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5b0c1a2bab449b869c4be20c8944c5, user_id=9e8d1b657bdd416199d0586c2925272e, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:08.008928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:08.018230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:08.153376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 909/1828] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:07 2022] PUT /identity/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5/users/9e8d1b657bdd416199d0586c2925272e/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:08.157906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:08.177700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29758326-ae38-4f23-931a-d867566ab073 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:08.283052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:08.284045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:08.284223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:08.284784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:08.285677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:08.290073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920601bb-c02c-4b6d-b110-c14635ecbfe1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:08.321174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 920/1829] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:08.325820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:08.346110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b52c-529c-4ec4-b911-b1d6c87504b6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:08.360033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:08.360943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:08.366142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:08.367064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:08.378781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] PATH_INFO: `/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5/users/9e8d1b657bdd416199d0586c2925272e/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:08.378781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5b0c1a2bab449b869c4be20c8944c5, user_id=9e8d1b657bdd416199d0586c2925272e, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:08.378781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:08.399440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 910/1830] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:08 2022] PUT /identity/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5/users/9e8d1b657bdd416199d0586c2925272e/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:08.401943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:08.402290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:08.402590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:08.446113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] MFA Rules not processed for user `9e8d1b657bdd416199d0586c2925272e`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:08.447489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:08.556702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 921/1831] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:08.579195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:08.582791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:08.669801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b71fa-a75d-487f-a303-4f102dbe9d6b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:08.673692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:08.674302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:08.674575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:08.674845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:08.685455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:08.687077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:08.702186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 911/1832] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:10.654875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:10.665281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e45f-d427-4293-86ba-f7abeaada212 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:10.677017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8d1b657bdd416199d0586c2925272e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5b0c1a2bab449b869c4be20c8944c5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:10.678040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:10.678483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:10.679546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:10.680387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:10.680836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:10.682600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:10.705392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 922/1833] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:10.715990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:10.727033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eb5c0-9db5-4bf3-9b76-d914e4466be4 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:10.732218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8d1b657bdd416199d0586c2925272e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5b0c1a2bab449b869c4be20c8944c5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:10.732773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:10.738346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:10.738346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:10.738346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:10.738346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:10.842068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 912/1834] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:10 2022] POST /identity/v3/projects => generated 353 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:10.861046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:10.885502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d634-2159-43f0-bf6a-8d3cff2bc475 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:10.990482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8d1b657bdd416199d0586c2925272e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5b0c1a2bab449b869c4be20c8944c5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:10.991162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:10.991455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:10.991742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:10.992439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:10.992767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:10.994419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:11.008985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 923/1835] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:11.062321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:11.079136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c1d51-15c3-4303-a435-0af30070d12b tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:11.084016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8d1b657bdd416199d0586c2925272e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5b0c1a2bab449b869c4be20c8944c5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:11.084655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:11.084931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:11.085202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] PATH_INFO: `/v3/projects/63f36ad91ba846079b6795333d547732` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:11.086312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorizing `identity:delete_project(project_id=63f36ad91ba846079b6795333d547732)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:11.088316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:11.429816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:11.447172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f36ad91ba846079b6795333d547732', 'request_id': 'req-ded2d78e-0e39-4e24-9e50-67960226e963'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:11.452749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f36ad91ba846079b6795333d547732', 'request_id': 'req-ded2d78e-0e39-4e24-9e50-67960226e963'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:11.640945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f36ad91ba846079b6795333d547732', 'request_id': 'req-ded2d78e-0e39-4e24-9e50-67960226e963'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:11.652772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] The token cache is being invalidate because project 63f36ad91ba846079b6795333d547732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:11.653060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:11.653723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:11.660008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 913/1836] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:11 2022] DELETE /identity/v3/projects/63f36ad91ba846079b6795333d547732 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:12.168668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:12.170571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:12.240575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ceeb-8cd0-4693-b98c-2ba3bdea8cc4 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:12.299064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:12.299710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:12.300667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:12.300953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] PATH_INFO: `/v3/users/f13e9d788e2244ea9de44c4b4b2eb447` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:12.302145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13e9d788e2244ea9de44c4b4b2eb447)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:12.304178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:12.613739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13e9d788e2244ea9de44c4b4b2eb447'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:12.614484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13e9d788e2244ea9de44c4b4b2eb447'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:12.626576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13e9d788e2244ea9de44c4b4b2eb447'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:12.633533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13e9d788e2244ea9de44c4b4b2eb447'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:12.695170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:12.696768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:12.710738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d78e-0e39-4e24-9e50-67960226e963 tempest-VolumeQuotaClassesTest-2053689289 tempest-VolumeQuotaClassesTest-2053689289-project-admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:12.715468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:12.716169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:12.716449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:12.716722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] PATH_INFO: `/v3/users/b185f216ff824a179ae5ac0951f7b2b3` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:12.717909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b185f216ff824a179ae5ac0951f7b2b3)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:12.719802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:12.788870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f2cd6d61-ab33-4c35-addc-d7dfbdb0bcbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13e9d788e2244ea9de44c4b4b2eb447'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:12.795325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 924/1837] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:12 2022] DELETE /identity/v3/users/f13e9d788e2244ea9de44c4b4b2eb447 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:12.799599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:12.799910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:12.800187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:12.819247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:12.820498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:12.880084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 925/1838] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:12.888740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:12.890301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:12.950187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a68a-159e-453b-af85-ccdf46586250 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:12.954360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:12.954982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:12.955261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:12.955536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:12.965939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:12.967424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c61c58-cb3c-4441-ac1d-0f6df0d846fc service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:12.977317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 926/1839] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:13.004143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b185f216ff824a179ae5ac0951f7b2b3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.125511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b185f216ff824a179ae5ac0951f7b2b3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.126146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b185f216ff824a179ae5ac0951f7b2b3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.134452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b185f216ff824a179ae5ac0951f7b2b3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.151793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b185f216ff824a179ae5ac0951f7b2b3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.158001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 914/1840] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:12 2022] DELETE /identity/v3/users/b185f216ff824a179ae5ac0951f7b2b3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:13.162595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:13.162906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:13.163205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:13.182746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:13.183915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:13.253747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 927/1841] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:13.263764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:13.265258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:13.336205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f581-6ff9-426e-a120-f15f6ac85d76 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:13.341628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:13.342344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:13.342344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:13.342604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:13.359694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:13.361817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:13.374578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 915/1842] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:13.707944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:13.709417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:13.722116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38c796-a9d9-433f-8e88-63d76fe32220 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:13.750358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:13.750358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:13.750358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:13.750358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] PATH_INFO: `/v3/projects/2c7f9b4bb6c34f91b03415295ecb6038` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:13.750891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7f9b4bb6c34f91b03415295ecb6038)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:13.752797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:13.979178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7f9b4bb6c34f91b03415295ecb6038', 'request_id': 'req-c90c86b5-d121-4554-9ca3-dd8482b9d51d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.985388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7f9b4bb6c34f91b03415295ecb6038', 'request_id': 'req-c90c86b5-d121-4554-9ca3-dd8482b9d51d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:13.992406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7f9b4bb6c34f91b03415295ecb6038', 'request_id': 'req-c90c86b5-d121-4554-9ca3-dd8482b9d51d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.007631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:14.199334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] The token cache is being invalidate because project 2c7f9b4bb6c34f91b03415295ecb6038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:14.200611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:14.202311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.202929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.203729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:14.208121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 928/1843] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:13 2022] DELETE /identity/v3/projects/2c7f9b4bb6c34f91b03415295ecb6038 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:14.209044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:14.210686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:14.218492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef92f40-faff-47e0-b1e6-56eba62625ea service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:14.292031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:14.292945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:14.293415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:14.294197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] PATH_INFO: `/v3/projects/ee16b597944f47e39637677fc62efa1e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:14.296028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c86b5-d121-4554-9ca3-dd8482b9d51d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:14.296423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee16b597944f47e39637677fc62efa1e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:14.300660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:14.306046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:14.306702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:14.306978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:14.307250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] PATH_INFO: `/v3/users/6fbf576dea5446ae828535a313ee13a6` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:14.308431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbf576dea5446ae828535a313ee13a6)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:14.316335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:14.438625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee16b597944f47e39637677fc62efa1e', 'request_id': 'req-cc7b9215-fed7-4e3a-8916-13895550b268'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.445013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee16b597944f47e39637677fc62efa1e', 'request_id': 'req-cc7b9215-fed7-4e3a-8916-13895550b268'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.554983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee16b597944f47e39637677fc62efa1e', 'request_id': 'req-cc7b9215-fed7-4e3a-8916-13895550b268'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.579401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbf576dea5446ae828535a313ee13a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.580555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbf576dea5446ae828535a313ee13a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.587459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] The token cache is being invalidate because project ee16b597944f47e39637677fc62efa1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:14.587770 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.594875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.598446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 916/1844] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:14 2022] DELETE /identity/v3/projects/ee16b597944f47e39637677fc62efa1e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:14.605126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:14.606582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbf576dea5446ae828535a313ee13a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.611436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:14.615666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbf576dea5446ae828535a313ee13a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.699442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b9215-fed7-4e3a-8916-13895550b268 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:14.706556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:14.707263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:14.707713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:14.708794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] PATH_INFO: `/v3/users/9e8d1b657bdd416199d0586c2925272e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:14.710537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8d1b657bdd416199d0586c2925272e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:14.713912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:14.736402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbf576dea5446ae828535a313ee13a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:14.751313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 929/1845] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:14 2022] DELETE /identity/v3/users/6fbf576dea5446ae828535a313ee13a6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:15.070255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8d1b657bdd416199d0586c2925272e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.075516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:15.078268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.175056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebff3-b092-40f0-b58e-9058f4e65f13 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:15.180578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:15.181226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:15.181506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:15.181915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] PATH_INFO: `/v3/users/c982237cf2c5473ca61ca26c1108f379` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:15.183471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c982237cf2c5473ca61ca26c1108f379)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:15.185744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:15.288480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8d1b657bdd416199d0586c2925272e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.291139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8d1b657bdd416199d0586c2925272e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.315929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8d1b657bdd416199d0586c2925272e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.346505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c5258685-7ad5-4f7b-add4-0c2e4d0b2b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8d1b657bdd416199d0586c2925272e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.357952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 917/1846] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:14 2022] DELETE /identity/v3/users/9e8d1b657bdd416199d0586c2925272e => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:15.394174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68064-a0fd-45bf-a3ab-82fb4cc2bee7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:15.394174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68064-a0fd-45bf-a3ab-82fb4cc2bee7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:15.394174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68064-a0fd-45bf-a3ab-82fb4cc2bee7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:15.407547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 918/1847] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:21:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:21:15.432517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:15.432517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:15.432517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:15.453016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:15.454340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.538723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 919/1848] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:15.562958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c982237cf2c5473ca61ca26c1108f379'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.564805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c982237cf2c5473ca61ca26c1108f379'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.567199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:15.568684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.591821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb25c06-db0b-4d49-af4d-2a00ecab8002 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:15.600508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:15.601527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:15.602093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:15.602587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:15.630483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c982237cf2c5473ca61ca26c1108f379'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.631097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:15.638384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f21cf6-a8de-4176-ac2b-df04cb3ef33c service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:15.638732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c982237cf2c5473ca61ca26c1108f379'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.659250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 920/1849] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:15.679985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:15.771770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c982237cf2c5473ca61ca26c1108f379'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:15.789290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 930/1850] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:15 2022] DELETE /identity/v3/users/c982237cf2c5473ca61ca26c1108f379 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:15.793820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:15.794110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:15.794385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:15.817883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:15.817883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.855320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:15.857631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.919970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 921/1851] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:15.930116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:15.932472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:15.991660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64241dd-b43d-4931-aad5-f6edeed5fc1c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:16.010327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:16.011245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:16.011779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:16.012228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] PATH_INFO: `/v3/projects/15d272eb9854425b8eb552ba4b245f2a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:16.013577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d272eb9854425b8eb552ba4b245f2a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:16.015685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:16.022849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71eb9-e4c0-4167-9e47-c9b388df4676 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:16.031303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:16.033217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:16.033217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:16.033217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:16.047043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:16.050808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:16.063089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 922/1852] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:17.588889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:17.591282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:17.607740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33961-b453-44b7-bc0b-443e2c65971e service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:17.618299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d272eb9854425b8eb552ba4b245f2a', 'request_id': 'req-f78d483e-9edc-4346-af76-bc0703a67c71'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.640478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d272eb9854425b8eb552ba4b245f2a', 'request_id': 'req-f78d483e-9edc-4346-af76-bc0703a67c71'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.668081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d272eb9854425b8eb552ba4b245f2a', 'request_id': 'req-f78d483e-9edc-4346-af76-bc0703a67c71'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.757107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:17.757782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:17.758022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:17.758344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] PATH_INFO: `/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:17.773095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5b0c1a2bab449b869c4be20c8944c5)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:17.774958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:17.830029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] The token cache is being invalidate because project 15d272eb9854425b8eb552ba4b245f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:17.830324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.832536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f78d483e-9edc-4346-af76-bc0703a67c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.839536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 931/1853] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:15 2022] DELETE /identity/v3/projects/15d272eb9854425b8eb552ba4b245f2a => generated 0 bytes in 1982 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:17.842876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea5818-a52c-41ba-822d-871d903c8b39 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:17.843174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea5818-a52c-41ba-822d-871d903c8b39 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:17.843452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea5818-a52c-41ba-822d-871d903c8b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:17.903916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-48ea5818-a52c-41ba-822d-871d903c8b39 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:17.905367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-48ea5818-a52c-41ba-822d-871d903c8b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:17.915154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 932/1854] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:17.923165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:17.926243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:17.926243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:17.944709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5b0c1a2bab449b869c4be20c8944c5', 'request_id': 'req-30b91bbd-dc51-4989-a560-8220f5b4187f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.954342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5b0c1a2bab449b869c4be20c8944c5', 'request_id': 'req-30b91bbd-dc51-4989-a560-8220f5b4187f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:17.969264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:17.971894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.034597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 933/1855] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:18.039317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.048618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29edf1d-aec9-4be7-b42d-0837d8d00064 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.053825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.054427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.054709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.054984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.055768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:18.056091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.057624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5b0c1a2bab449b869c4be20c8944c5', 'request_id': 'req-30b91bbd-dc51-4989-a560-8220f5b4187f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.058014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.076735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 934/1856] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:21:18.076927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] The token cache is being invalidate because project 4e5b0c1a2bab449b869c4be20c8944c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:18.077536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.078237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.078587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-30b91bbd-dc51-4989-a560-8220f5b4187f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.080220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.080745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 923/1857] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:17 2022] DELETE /identity/v3/projects/4e5b0c1a2bab449b869c4be20c8944c5 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:18.085969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.086259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.086537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.096284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f572-0e6b-49b4-be87-ec5a653b22c9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.102951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.102951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.103313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.103589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.104433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.106453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a527cf-b77c-4184-9f48-49e471ef4066 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.108337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:18.109867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.125527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 924/1858] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:18.125527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.125527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.139671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897460b-1adb-4a7b-8ac0-d8819f860663 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.152383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.153366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.153366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.153574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] PATH_INFO: `/v3/projects/7d88ef34b22242b3afb6a277772cc88c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.154840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d88ef34b22242b3afb6a277772cc88c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.158195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.196898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 935/1859] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:18.198261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.198564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.198854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.238955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:18.244743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.260590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d88ef34b22242b3afb6a277772cc88c', 'request_id': 'req-fce7c340-8b5e-48be-84a8-6718fbad9f0b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.266653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d88ef34b22242b3afb6a277772cc88c', 'request_id': 'req-fce7c340-8b5e-48be-84a8-6718fbad9f0b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.307494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 936/1860] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:18.308290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.317437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ada04-197b-4882-8372-1ab8beca2a04 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.324074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.324830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.327962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.327962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.327962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.328222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.354851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d88ef34b22242b3afb6a277772cc88c', 'request_id': 'req-fce7c340-8b5e-48be-84a8-6718fbad9f0b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.367975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] The token cache is being invalidate because project 7d88ef34b22242b3afb6a277772cc88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:18.367975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.368181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.369957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 925/1861] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:18 2022] DELETE /identity/v3/projects/7d88ef34b22242b3afb6a277772cc88c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:18.373308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.374719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.389796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c340-8b5e-48be-84a8-6718fbad9f0b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.396366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.398119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.398451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.398784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.399465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:18.399774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.401439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.421163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 926/1862] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:18.422184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.424175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.438556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6273-f9a4-4bfa-ae22-d0d12b669921 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.448609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.449703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.450102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.450419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] PATH_INFO: `/v3/users/916a7b57790e43beb7b299a29e725609` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.451681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916a7b57790e43beb7b299a29e725609)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.453704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.454131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 937/1863] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:18.455135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.466098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97de9-6124-4e1d-a4ff-0ae79d8604a1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.473818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.474689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.475121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.475540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.476223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.478570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.620235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 938/1864] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/projects => generated 407 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:18.622123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.624891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.671102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916a7b57790e43beb7b299a29e725609'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.735179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafff3-12f2-4e60-b0bb-cdbb7a74aa97 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.740055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.740784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.741256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.741618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.742371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.744098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.757694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 939/1865] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:18.763557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.770561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2a77-38aa-468a-8f8e-71107008dcac admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:18.775341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:18.776026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:18.776307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:18.776579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:18.777214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:18.779133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:18.828666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916a7b57790e43beb7b299a29e725609'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.829514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916a7b57790e43beb7b299a29e725609'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.838467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916a7b57790e43beb7b299a29e725609'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.852447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916a7b57790e43beb7b299a29e725609'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:18.860274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 927/1866] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:18 2022] DELETE /identity/v3/users/916a7b57790e43beb7b299a29e725609 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:18.861456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.862970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.918708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 940/1867] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:18 2022] POST /identity/v3/users => generated 304 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:18.921046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:18.928518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:18.964434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372792-44e4-4c54-b2cc-cbebef8066d2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.002618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e0ef4-f9b3-4430-a40e-aed5eb1982a9 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.014319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.016546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.016854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.017135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.017867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.020461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.036383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 928/1868] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:19.037341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:19.048268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60910f3-cb3c-43e7-9f21-389f1b16a819 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.062843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.064179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.064459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.064732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] PATH_INFO: `/v3/projects/b8a7986d580d4511957562118685cd9a/users/c248518c95234052bc979b9270e1f57b/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.077541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.079772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.082154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.083601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.085120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.086918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a7986d580d4511957562118685cd9a, user_id=c248518c95234052bc979b9270e1f57b, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.087970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.090059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d4dc-8a66-4ace-803c-98cd2167500f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.105974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 941/1869] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:19.106860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:19.117657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a672f63-3d6d-42c7-b90d-1c3b60f1b86d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.123256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.123919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.124252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.124651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.125772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.127454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.140796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 942/1870] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:19.145785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:19.156084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfba800-4d94-4966-be76-d83d529a8c4e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.167360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.169112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.170653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.170653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] PATH_INFO: `/v3/projects/de82c6d4af9e4516991e2816ad2699d8/users/601207a5abf544d9b99d477337f14026/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.185838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de82c6d4af9e4516991e2816ad2699d8, user_id=601207a5abf544d9b99d477337f14026, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.188556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8b254-9928-449a-98ae-0f999fbe597e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.319275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 929/1871] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:19 2022] PUT /identity/v3/projects/b8a7986d580d4511957562118685cd9a/users/c248518c95234052bc979b9270e1f57b/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:19.332898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.333113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.333414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.360290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] MFA Rules not processed for user `c248518c95234052bc979b9270e1f57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:19.361503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:19.412505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 943/1872] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:19 2022] PUT /identity/v3/projects/de82c6d4af9e4516991e2816ad2699d8/users/601207a5abf544d9b99d477337f14026/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:19.413268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.413528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.413802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.445217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] MFA Rules not processed for user `601207a5abf544d9b99d477337f14026`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:19.446863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:19.451812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 930/1873] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:19.452695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:19.454634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:19.541093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c309d-3f13-4600-b066-5c96cd2a5cd3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.548520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 944/1874] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:19.548520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:19.637969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2aa9d-ac98-4b1b-8318-021b9abb93c2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:19.642584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.643042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:19.643545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.644492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:19.644781 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.645058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.645706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.647738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.654224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:19.660204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] PATH_INFO: `/v3/users/e6b3dde2253240c18d33fba23b8329ed` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:19.660204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b3dde2253240c18d33fba23b8329ed)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:19.660204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:19.921540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 945/1875] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:19 2022] POST /identity/v3/projects => generated 387 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:19.921540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.005140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97665b-a85a-49cb-828e-0db3197397e2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.012360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.013849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.014292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.014575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.015658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.020338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.082689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b3dde2253240c18d33fba23b8329ed'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.195289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 946/1876] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:19 2022] POST /identity/v3/projects => generated 409 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:20.196101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.197669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.222736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b3dde2253240c18d33fba23b8329ed'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.223713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b3dde2253240c18d33fba23b8329ed'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.231750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b3dde2253240c18d33fba23b8329ed'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.245173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b3dde2253240c18d33fba23b8329ed'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.251934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 931/1877] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:19 2022] DELETE /identity/v3/users/e6b3dde2253240c18d33fba23b8329ed => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:20.253747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.255249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.274552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea32fe0-80f6-48d8-93be-d3004277cb35 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.344666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.345337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.345616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.348237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] PATH_INFO: `/v3/projects/49bbf675d88d4377b8c21d5384b789f7` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.348237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bbf675d88d4377b8c21d5384b789f7)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.348408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e734-3d6b-4e19-964b-6a10e2e653bf admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.349163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.353606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.354532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.354987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.355430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.356263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.358360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.538973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 932/1878] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:20 2022] POST /identity/v3/users => generated 300 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:20.540399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.541655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.554371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce088ef-7c60-4e72-b2ee-2b43c68cef41 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.559451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.560090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.560390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.560665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.561349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.563272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a3228-c819-4b8f-a03f-6143437f22e5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.614668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bbf675d88d4377b8c21d5384b789f7', 'request_id': 'req-af69deec-a9b6-483e-aee0-260f9a589151'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.620121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bbf675d88d4377b8c21d5384b789f7', 'request_id': 'req-af69deec-a9b6-483e-aee0-260f9a589151'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.626205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bbf675d88d4377b8c21d5384b789f7', 'request_id': 'req-af69deec-a9b6-483e-aee0-260f9a589151'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.706758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 933/1879] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:20 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:20.706758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.706758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.706758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.725810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:20.727008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.742804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] The token cache is being invalidate because project 49bbf675d88d4377b8c21d5384b789f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:20.743353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.743693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:20.745492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 947/1880] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:20 2022] DELETE /identity/v3/projects/49bbf675d88d4377b8c21d5384b789f7 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:20.746188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.747601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.792646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 934/1881] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:20.793767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.795343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.812333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf893cc2-8394-4f24-8732-ce26205ecb06 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.817046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.817278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69deec-a9b6-483e-aee0-260f9a589151 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.817645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.817923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.818240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.818873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.820501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891fb0f-04f6-4bcc-8c7d-8de173a062bc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.821844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.822454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.822736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.823007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.823643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.825559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.833749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 935/1882] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:20.833749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.833749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.833749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.836237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 948/1883] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:20.836863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.840062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.853369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:20.854554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:20.863151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 936/1884] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:20.863631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.872884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140b5b-75ba-4630-bf08-8bf2f85d0c49 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.885355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.885355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.885355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.885355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] PATH_INFO: `/v3/projects/a6e50f1a125f4639ae0fc8e76153a826/users/e95cd113e0e74ecba45b2ae440f49641/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.893158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e50f1a125f4639ae0fc8e76153a826, user_id=e95cd113e0e74ecba45b2ae440f49641, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.896818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483600f-5838-4875-b56b-8096c2c5c5c6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.904035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567e7f1-3556-4cf3-b4f6-6c9726129405 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.907653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.908292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.908564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.908833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.918641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.928791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:20.929178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 949/1885] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:20.929959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:20.939049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debaaca-5878-4f53-b408-76d993367c62 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:20.943847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:20.944694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:20.944989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:20.945265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] PATH_INFO: `/v3/projects/557ddf027cdb449f8893e7a1621cb99e/users/88f9749c5cb849928032e282614e7a92/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:20.960693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557ddf027cdb449f8893e7a1621cb99e, user_id=88f9749c5cb849928032e282614e7a92, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:20.963368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.041173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 937/1886] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:20 2022] PUT /identity/v3/projects/a6e50f1a125f4639ae0fc8e76153a826/users/e95cd113e0e74ecba45b2ae440f49641/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:21.042523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.042863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.043115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.064223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:21.065460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:21.103838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:21.111067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 950/1887] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:20 2022] PUT /identity/v3/projects/557ddf027cdb449f8893e7a1621cb99e/users/88f9749c5cb849928032e282614e7a92/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:21.111995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.126991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 938/1888] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:21.128571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.139420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be0720-4fea-4fa2-8aeb-5de6e252750b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.192094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0fb25-f466-4aeb-8b3d-d1580da735e9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.198230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.198624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.199084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.199361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.200200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.210774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.210774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.210774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.210774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.213291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.213712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.215243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.230031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 939/1889] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:21.230031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.235559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 951/1890] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:21.236738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.243842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4468ba-004b-4171-a494-df228e1affc0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.249107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.249948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.250303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.250677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.251404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:21.251727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89528744-7ad5-4c84-90a7-f6d7aea43e00 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.251843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.254614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.258695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.260064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.260418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.260872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] PATH_INFO: `/v3/projects/557ddf027cdb449f8893e7a1621cb99e/users/88f9749c5cb849928032e282614e7a92/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.263052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557ddf027cdb449f8893e7a1621cb99e, user_id=88f9749c5cb849928032e282614e7a92, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.265813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.270213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 940/1891] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:21.270213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.284133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 952/1892] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:21 2022] PUT /identity/v3/projects/557ddf027cdb449f8893e7a1621cb99e/users/88f9749c5cb849928032e282614e7a92/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:21.294514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d46b5-1811-42d4-b3d2-349c707ce217 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.294640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.376363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.377291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.377913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.378350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] PATH_INFO: `/v3/projects/a6e50f1a125f4639ae0fc8e76153a826/users/e95cd113e0e74ecba45b2ae440f49641/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.380578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e50f1a125f4639ae0fc8e76153a826, user_id=e95cd113e0e74ecba45b2ae440f49641, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.384674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a62ce-1a29-49e8-96c2-4e68120fd64d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.389620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36216d96-f7c1-490b-bda0-49d75183f4d9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.395612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.396281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.396551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.396827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.397461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.404786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 941/1893] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:21 2022] PUT /identity/v3/projects/a6e50f1a125f4639ae0fc8e76153a826/users/e95cd113e0e74ecba45b2ae440f49641/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:21.404988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.405073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4736a8-bfee-4aba-bde5-d0275ea020f2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.405073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4736a8-bfee-4aba-bde5-d0275ea020f2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.405073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4736a8-bfee-4aba-bde5-d0275ea020f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.438603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cf4736a8-bfee-4aba-bde5-d0275ea020f2 None None] MFA Rules not processed for user `88f9749c5cb849928032e282614e7a92`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:21.445234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cf4736a8-bfee-4aba-bde5-d0275ea020f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:21.516876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 942/1894] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:21.519118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.519682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.520222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.550369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 953/1895] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:21 2022] POST /identity/v3/projects => generated 375 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:21.551153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.554290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:21.564183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] MFA Rules not processed for user `e95cd113e0e74ecba45b2ae440f49641`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:21.565634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:21.663115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ad10-8b1e-4882-a737-942fccd4901a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.669658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.670532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.670814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.671090 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] PATH_INFO: `/v3/projects/ba464c95205f423bb3b5f55a5e4dfd63` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.672410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba464c95205f423bb3b5f55a5e4dfd63)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.674335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.676873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 943/1896] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:21.677913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.679654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:21.715846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:21.760639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030936fb-65bb-4a66-be92-74d61b952f68 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.766144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:21.767134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:21.767134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:21.767476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:21.841226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba464c95205f423bb3b5f55a5e4dfd63', 'request_id': 'req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:21.851876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba464c95205f423bb3b5f55a5e4dfd63', 'request_id': 'req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:21.863132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba464c95205f423bb3b5f55a5e4dfd63', 'request_id': 'req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:21.870309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:21.871893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:21.894521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 944/1897] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:21.896624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.914816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66da81-add2-4684-896c-2a603bc7fd3b service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:21.977892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] The token cache is being invalidate because project ba464c95205f423bb3b5f55a5e4dfd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:21.978314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:21.979226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:21.981058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 954/1898] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:21 2022] DELETE /identity/v3/projects/ba464c95205f423bb3b5f55a5e4dfd63 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:21.981923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:21.986581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.011295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.011964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.012244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.012516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.013311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.016834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.068415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3156eb-cd6c-42b4-b5f5-04afbcc5b779 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.072485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.073350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.073809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.074178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.076136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.162286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.164578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.181519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 955/1899] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:22.182869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.184471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.202526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7df14-f0d1-4a16-bc7f-41cdfa9ecd00 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.213228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.214148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.215110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.215110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] PATH_INFO: `/v3/users/c6ed29506969418aac8d3f7ba690fd51` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.217018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ed29506969418aac8d3f7ba690fd51)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.219885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.286530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 945/1900] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:21 2022] POST /identity/v3/users => generated 288 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:22.291683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.293702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.326344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c58b8-a926-4889-89b7-c547bd231929 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.331451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.332110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.332394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.332671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.333323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.334971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.351612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 946/1901] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:22.356613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.370008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd82b-6881-4b38-b53c-a9aaaded626b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.374913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.375534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.375812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.376115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.376928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.379166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.397576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 947/1902] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:22.397576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.408332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428db52-f111-43c2-a918-1d7562174960 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.413648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.414984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.415338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.415829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] PATH_INFO: `/v3/projects/c8136e1fd1e14383be9330627b0f1111/users/7afbfa44402646d4a9d3e8c5cb104214/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.434026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8136e1fd1e14383be9330627b0f1111, user_id=7afbfa44402646d4a9d3e8c5cb104214, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:22.436899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51362143-a160-425f-9e02-bba5fc77c41f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:22.521230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ed29506969418aac8d3f7ba690fd51'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:22.522658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ed29506969418aac8d3f7ba690fd51'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:22.536509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ed29506969418aac8d3f7ba690fd51'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:22.559658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ed29506969418aac8d3f7ba690fd51'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:22.561359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 948/1903] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:22 2022] PUT /identity/v3/projects/c8136e1fd1e14383be9330627b0f1111/users/7afbfa44402646d4a9d3e8c5cb104214/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:22.573922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.574453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.574756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.637856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] MFA Rules not processed for user `7afbfa44402646d4a9d3e8c5cb104214`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:22.639687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.731344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ed29506969418aac8d3f7ba690fd51'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:22.740531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 956/1904] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:22 2022] DELETE /identity/v3/users/c6ed29506969418aac8d3f7ba690fd51 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:22.758288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 949/1905] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:22.774966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.777424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.812344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:22.814162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:22.908736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf2d5b-436b-4d5d-a0b5-499745285e6c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.918897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fc358-254e-4068-b1d7-cd759e43a571 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:22.919034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.919034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.919034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.919034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.993178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:22.993844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:22.996547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:22.996547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:22.998762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:23.025940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:23.027678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:23.057537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 957/1906] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:23.100483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:23.104893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae17569-565e-4a9b-a6b0-3b18b24f66b1 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:23.120879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 950/1907] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:23.391742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:23.393388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:23.472745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d76300-61fa-4a16-9ab4-0eb476d26d9c service glance] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:23.477604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:23.478269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:23.478581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:23.478806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] PATH_INFO: `/v3/projects/ef57419a1206491cad5d733a9d818540` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:23.480252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef57419a1206491cad5d733a9d818540)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:23.482203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:23.649011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef57419a1206491cad5d733a9d818540', 'request_id': 'req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:23.654763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef57419a1206491cad5d733a9d818540', 'request_id': 'req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:23.660993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef57419a1206491cad5d733a9d818540', 'request_id': 'req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:23.778244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:23.793633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] The token cache is being invalidate because project ef57419a1206491cad5d733a9d818540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:23.793633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:23.793633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdfc0551-ce97-46f0-8b4d-e3bffaddfc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:23.793633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 958/1908] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:23 2022] DELETE /identity/v3/projects/ef57419a1206491cad5d733a9d818540 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:23.793971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:23.793971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:23.793971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:23.814752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:23.816800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:23.876751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 951/1909] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:23.886389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:23.886669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:23.886945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:23.907006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:23.908184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:23.916522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 959/1910] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:23.920515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:23.930817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacdeab-db43-4c99-a65d-5a41b1313cab None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:23.936564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:23.938029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:23.938371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:23.938705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:23.939427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:23.939801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:23.942312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:23.954864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 952/1911] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:23.959475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:23.969129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178bbc-2c5e-40ad-a9f5-cc1d7787ee60 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:23.973996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:23.974611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:23.974893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:23.975162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:23.975804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:23.977767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:24.211014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 960/1912] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:23 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:24.215104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:24.228109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e215-25c2-445f-852c-11009bd62ee4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:24.307084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:24.307884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.308064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.308547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.309183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:24.311149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:24.365171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:24.428775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 953/1913] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:24 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:24.430014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:24.440851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e194978-9bac-4ded-b0fc-395a53de990a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:24.445833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:24.446446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.446725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.446996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.447636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:24.449448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:24.475143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:24.481017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 961/1914] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:24.501938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8793dd-7800-4337-9f55-faf9c93c7658 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:24.507022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:24.507742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.508839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.509209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.510083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:24.512065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce8095-ff34-47b5-9c20-ebdfe31bbd14 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:24.529128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:24.529846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 954/1915] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:24.540547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fec2a-fbd7-4daa-af02-770803d13c88 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:24.546112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:24.547950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.548371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.548653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] PATH_INFO: `/v3/projects/9b4d5ce638d74ba88d0d2c8cff37c3e0/users/62b70f76c2f04e8684c5aa5dd3787d80/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.572321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4d5ce638d74ba88d0d2c8cff37c3e0, user_id=62b70f76c2f04e8684c5aa5dd3787d80, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:24.574980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:24.717661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 962/1916] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:24 2022] PUT /identity/v3/projects/9b4d5ce638d74ba88d0d2c8cff37c3e0/users/62b70f76c2f04e8684c5aa5dd3787d80/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:24.728298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.728617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.728988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.765376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] MFA Rules not processed for user `62b70f76c2f04e8684c5aa5dd3787d80`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:24.766845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:24.858761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 955/1917] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:24.864353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:24.964323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a46de-30b9-4407-a064-8a845cf22f12 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:24.983637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:24.984834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:24.986084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:24.986532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:24.987530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:24.989704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fa50a-d03f-400c-b6be-92ff9593cc94 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.184476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 963/1918] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:24 2022] POST /identity/v3/projects => generated 387 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:25.187324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.209601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb45-d4e8-4b6f-aeda-a073a6f18273 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.209601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb45-d4e8-4b6f-aeda-a073a6f18273 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.212183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb45-d4e8-4b6f-aeda-a073a6f18273 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.216798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c8ec-6ff6-4620-a14e-d7aa73e3df0a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.218221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 964/1919] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:21:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:21:25.228532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.229036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.229385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.296816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:25.298543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:25.311254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.313890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.313890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.313890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.315293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:25.319236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.419997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 965/1920] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:25.508734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 956/1921] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:25 2022] POST /identity/v3/users => generated 294 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:25.513530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.525852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf0ee6-b6fe-478e-8ecd-acaf1d91d3eb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.531941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.532634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.532982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.533254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.534528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:25.536482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.551318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 966/1922] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:25.556048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.574032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb0b8-0f0b-4018-9252-d7de693d6fce admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.582812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.584536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:25.586021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.586947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.588268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.588638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.589398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:25.591767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.621752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3cce7-2b78-4529-9776-aa785de927f2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.633588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 957/1923] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:25.635610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.636525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.636981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.637406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] PATH_INFO: `/v3/users/c248518c95234052bc979b9270e1f57b` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.640672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c248518c95234052bc979b9270e1f57b)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:25.645520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.660084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.691545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a621835-a17a-4e5e-abd1-1392afb42b6a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.700597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.701512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.702091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.702386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] PATH_INFO: `/v3/projects/5723b400d8f544e08688ddba95002f4f/users/a4ea08794d6042d0bc63043c441b8e6b/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:25.723043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5723b400d8f544e08688ddba95002f4f, user_id=a4ea08794d6042d0bc63043c441b8e6b, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:25.729306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:25.881920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 958/1924] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:25 2022] PUT /identity/v3/projects/5723b400d8f544e08688ddba95002f4f/users/a4ea08794d6042d0bc63043c441b8e6b/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:25.883486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:25.888332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:25.989969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9315c-2064-40d0-9219-90eede5586ed admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:25.995884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:25.996615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:25.996896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:25.997166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.006675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.008320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fb6c3-b030-4607-8416-c5c0ca52be10 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.022077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 959/1925] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:26.025249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.025563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.025857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.050875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c248518c95234052bc979b9270e1f57b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:26.052478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c248518c95234052bc979b9270e1f57b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:26.062321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] MFA Rules not processed for user `a4ea08794d6042d0bc63043c441b8e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:26.063613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:26.076265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c248518c95234052bc979b9270e1f57b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:26.084773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c248518c95234052bc979b9270e1f57b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:26.141357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 960/1926] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:26.148660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.221935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2752f9e-9bf2-40a8-8fc5-e4a590d6959c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.222757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f81f47f9-daf8-46a9-b9cc-3ace5894742e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c248518c95234052bc979b9270e1f57b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:26.226914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.227549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.227827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.228193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.228901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.231428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 967/1927] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:25 2022] DELETE /identity/v3/users/c248518c95234052bc979b9270e1f57b => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:26.232209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.235269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.235552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.235824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.256249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:26.257428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:26.323288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 968/1928] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:26.333093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.334635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:26.384363 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 961/1929] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:26 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:26.386274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.389677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:26.408591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d45a7-7d84-4bff-8cea-c36ad74ceab5 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.461975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.462588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.462892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.463135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.467535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52eaaf-1bdd-428f-86a4-1685e013af77 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.473137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.473274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.473744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.474019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.474311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.474630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.474941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.476889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.492044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 969/1930] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:26.587637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 962/1931] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:26 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:26.589214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.603899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062eec5-9a66-4480-a7bc-7c8d974cc719 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.611601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.613175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.613610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.614048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.615111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.617475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:26.617977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.629672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 970/1932] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:26.634028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.644744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3801-8fd9-4b4c-b6b3-e05645a2536d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.649559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.650405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.650642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.650915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] PATH_INFO: `/v3/projects/8e6f56acce0c4735838ce64b4c9ee051/users/357ca573cf4b45939d5665a2732ff63d/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.667238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6f56acce0c4735838ce64b4c9ee051, user_id=357ca573cf4b45939d5665a2732ff63d, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.670088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.766790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 963/1933] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:26 2022] PUT /identity/v3/projects/8e6f56acce0c4735838ce64b4c9ee051/users/357ca573cf4b45939d5665a2732ff63d/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:26.772097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.797515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2a6f-f2fe-49ba-b2af-c14ac1fc3412 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.800356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:26.881507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.882427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.882892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.883516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.884402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.886257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa969e-03d3-4b72-a071-fe0669a865f4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.900265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 971/1934] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:26.904073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:26.915768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d9aa8-b587-41a8-8ad5-7d52c4db22b3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:26.921535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:26.922314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.922651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.922995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] PATH_INFO: `/v3/projects/8e6f56acce0c4735838ce64b4c9ee051/users/357ca573cf4b45939d5665a2732ff63d/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:26.925864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6f56acce0c4735838ce64b4c9ee051, user_id=357ca573cf4b45939d5665a2732ff63d, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:26.928928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:26.956810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 964/1935] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:26 2022] PUT /identity/v3/projects/8e6f56acce0c4735838ce64b4c9ee051/users/357ca573cf4b45939d5665a2732ff63d/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:26.967774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:26.968239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:26.970148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:27.009931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] MFA Rules not processed for user `357ca573cf4b45939d5665a2732ff63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:27.011800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:27.032189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:27.036204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:27.120932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 972/1936] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:27.132727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:27.134397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:27.134803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf62438-93cb-448e-a88d-b81b7d21c3bb admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:27.141262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:27.141906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:27.142180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:27.142451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] PATH_INFO: `/v3/projects/b8a7986d580d4511957562118685cd9a` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:27.143637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a7986d580d4511957562118685cd9a)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:27.145593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:27.207758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd11b4-829f-4083-9adc-597cf0e28e48 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:27.211693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:27.212333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:27.212608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:27.212877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:27.222895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:27.224715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:27.236394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 973/1937] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:27.291290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a7986d580d4511957562118685cd9a', 'request_id': 'req-42be0954-48ef-4e02-af8b-419ce0ba16f4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.297209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a7986d580d4511957562118685cd9a', 'request_id': 'req-42be0954-48ef-4e02-af8b-419ce0ba16f4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.405554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a7986d580d4511957562118685cd9a', 'request_id': 'req-42be0954-48ef-4e02-af8b-419ce0ba16f4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.417746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] The token cache is being invalidate because project b8a7986d580d4511957562118685cd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:27.418037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.418715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.420280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 965/1938] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:27 2022] DELETE /identity/v3/projects/b8a7986d580d4511957562118685cd9a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:27.424732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:27.436977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df94b-05c3-4bed-a46a-08264ef0d986 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:27.514310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:27.514959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:27.515225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:27.515523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] PATH_INFO: `/v3/users/88f9749c5cb849928032e282614e7a92` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:27.516647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f9749c5cb849928032e282614e7a92)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:27.519229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:27.851205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f9749c5cb849928032e282614e7a92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.852037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f9749c5cb849928032e282614e7a92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.871915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f9749c5cb849928032e282614e7a92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.876945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f9749c5cb849928032e282614e7a92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.985256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f9749c5cb849928032e282614e7a92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:27.996025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 974/1939] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:27 2022] DELETE /identity/v3/users/88f9749c5cb849928032e282614e7a92 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:28.520922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:28.522602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:28.556458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0954-48ef-4e02-af8b-419ce0ba16f4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:28.622789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:28.623846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:28.624403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:28.624919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] PATH_INFO: `/v3/projects/557ddf027cdb449f8893e7a1621cb99e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:28.626541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557ddf027cdb449f8893e7a1621cb99e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:28.629078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:28.653861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:28.655153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:28.723984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e9c8f-cd06-416c-b200-52e484db41be admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:28.727593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:28.728308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:28.728595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:28.728875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:28.730738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:28.761292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557ddf027cdb449f8893e7a1621cb99e', 'request_id': 'req-c0acf1da-4b13-466b-8791-35b96d04dea1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:28.766855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557ddf027cdb449f8893e7a1621cb99e', 'request_id': 'req-c0acf1da-4b13-466b-8791-35b96d04dea1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:28.791410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:28.793083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b0881-59a2-4584-bef0-54ab374696a2 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:28.803258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 975/1940] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:28.905218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557ddf027cdb449f8893e7a1621cb99e', 'request_id': 'req-c0acf1da-4b13-466b-8791-35b96d04dea1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:28.921027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] The token cache is being invalidate because project 557ddf027cdb449f8893e7a1621cb99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:28.921465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:28.922909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c0acf1da-4b13-466b-8791-35b96d04dea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:28.925209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 966/1941] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:28 2022] DELETE /identity/v3/projects/557ddf027cdb449f8893e7a1621cb99e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:28.930045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:28.931833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:28.935088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:28.982889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:28.984514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:29.058425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 976/1942] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:29.066271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.066559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.066848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.088129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:29.089811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:29.099572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 967/1943] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:29.103838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.117332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95611b33-e81a-408e-8b78-07a6b6623326 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.122155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.122782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.123083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.123383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.124051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:29.124391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.126262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.151307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 977/1944] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:29.152629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.163005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c135a-b5fe-4e6b-b6dc-8af668dd32c0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.167681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.168431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.168705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.168977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.169604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.171520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.278783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 968/1945] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:29 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:29.283173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.294010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b303c-7c43-4be4-9cf1-8d932775bbbb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.370961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.371553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.371829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.372161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.372803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.374817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.504682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 978/1946] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:29 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:29.515957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.527134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2dd55-148f-436f-9e82-f38c7a796953 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.532103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.532700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.533107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.533286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.533986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.535745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.549706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 969/1947] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:29.551035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.561348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41811-23b2-4ed3-b9bb-8c3d5b826bbd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.566616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.567226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.567515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.572512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.572512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.572512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985f2ea-1b1f-4f22-92b4-6d855a31617d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.585672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 979/1948] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:29.590733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.617462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee154cb-cca9-44aa-9702-de5b4ff974d2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.629849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.630596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.630973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.631369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] PATH_INFO: `/v3/projects/cf7d567a34404963a875cea01d9d8d1c/users/b1219819ed19468590c399e06ca8b1c4/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.655342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7d567a34404963a875cea01d9d8d1c, user_id=b1219819ed19468590c399e06ca8b1c4, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.658406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:29.788129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:29.795353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 970/1949] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:29 2022] PUT /identity/v3/projects/cf7d567a34404963a875cea01d9d8d1c/users/b1219819ed19468590c399e06ca8b1c4/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:29.801292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cf920-a959-41b9-b223-e9e213cb91c4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.802085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cf920-a959-41b9-b223-e9e213cb91c4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.802251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cf920-a959-41b9-b223-e9e213cb91c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.828005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-582cf920-a959-41b9-b223-e9e213cb91c4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:29.829238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-582cf920-a959-41b9-b223-e9e213cb91c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:29.897455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 980/1950] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:29.908086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:29.909819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:29.972561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad3f11-1826-4459-817d-1570f28a6fd3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:29.980928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:29.981555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:29.981828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:29.982096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:29.993070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:29.994634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:30.004893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 971/1951] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:30.067741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:31.473750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:31.474334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:31.474334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:31.505115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] MFA Rules not processed for user `b1219819ed19468590c399e06ca8b1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:31.506377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:31.575272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 981/1952] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:31.579497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:31.589542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9580fe4-32fc-495b-9a38-20de4d1ea3d4 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:31.594280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:31.596851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:31.596851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:31.596851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:31.596851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:31.597986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:31.738299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 972/1953] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:31 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:31.742519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:31.757650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f900f-15ad-44b9-8901-622eff3bcd50 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:31.828095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:31.828724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:31.829004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:31.829278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:31.829930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:31.832067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:31.981949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:32.025329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 982/1954] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:31 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:32.041128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:32.055043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b8ed9-02ec-4294-81cf-1e06fb79c6da admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:32.063410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:32.064070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:32.068386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:32.068386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:32.068386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:32.068818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:32.081244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 973/1955] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:32.085499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:32.096605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9736f-85da-48f5-8bca-a99ed46fbedb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:32.101748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:32.102868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:32.103135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:32.103524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] PATH_INFO: `/v3/projects/0975096b0cd04c4198e3117328d304b0/users/0af82ba96b5743be9cd44263fd0ca225/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:32.125888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0975096b0cd04c4198e3117328d304b0, user_id=0af82ba96b5743be9cd44263fd0ca225, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:32.129295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:32.215985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:32.302170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 983/1956] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:32 2022] PUT /identity/v3/projects/0975096b0cd04c4198e3117328d304b0/users/0af82ba96b5743be9cd44263fd0ca225/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:32.306373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:32.317328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7728de-e480-424c-8647-b27db6f26bb6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:32.389328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:32.401119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:32.401119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:32.401119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:32.401119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:32.401119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968de565-523d-40e6-9110-5303a21add0f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:32.403612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 974/1957] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:32.407888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:32.418742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575eab5-b245-492d-88af-4be8ac853256 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:32.423467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:32.424204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:32.425003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:32.425277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] PATH_INFO: `/v3/projects/0975096b0cd04c4198e3117328d304b0/users/0af82ba96b5743be9cd44263fd0ca225/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:32.427176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0975096b0cd04c4198e3117328d304b0, user_id=0af82ba96b5743be9cd44263fd0ca225, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:32.430017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:32.445367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 984/1958] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:32 2022] PUT /identity/v3/projects/0975096b0cd04c4198e3117328d304b0/users/0af82ba96b5743be9cd44263fd0ca225/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:33.109417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:34.359476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:34.359476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:34.359476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:34.391083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] MFA Rules not processed for user `0af82ba96b5743be9cd44263fd0ca225`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:34.392641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:34.423237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:34.548866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 975/1959] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:34.564213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:34.564213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:34.642715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f0bd2-d765-4cfe-b75f-273264d7be07 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:34.648318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:34.649025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:34.649627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:34.650159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:34.766497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:34.768641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:34.778936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 985/1960] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:36.308577 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:38.057685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:38.067829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b2f5-16f4-4978-b071-4b53d3cf7507 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:38.071501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:38.072600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:38.073320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:38.073320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:38.083322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:38.084845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:38.094697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 976/1961] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:38.140907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:38.143300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:38.154414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadc99f-66cc-4da6-9564-166baa06a897 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:38.158002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:38.159290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:38.159619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:38.159828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:38.161543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:38.223204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:38.224629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee9b2-6d42-4b20-a6b8-ffc845383281 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:38.233652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 986/1962] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:38.317504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:38.319080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:38.397848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef423b02-641c-4c8e-9a88-f627382e0fb6 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:38.402865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:38.403504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:38.403784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:38.404095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] PATH_INFO: `/v3/users/1eb26f9e1b28484fb3216feecba34b56` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:38.405297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb26f9e1b28484fb3216feecba34b56)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:38.407324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:38.559427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:38.660550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb26f9e1b28484fb3216feecba34b56'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:38.743588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb26f9e1b28484fb3216feecba34b56'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:38.744416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb26f9e1b28484fb3216feecba34b56'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:38.753767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb26f9e1b28484fb3216feecba34b56'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:38.777764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb26f9e1b28484fb3216feecba34b56'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:38.790531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 977/1963] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:38 2022] DELETE /identity/v3/users/1eb26f9e1b28484fb3216feecba34b56 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:38.795100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:38.795418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:38.796047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:38.816733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:38.819673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:38.883208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 987/1964] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:38.893706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:38.895186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:38.956831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069cff-22b9-4676-8c92-4e69d599e783 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:38.961241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:38.961914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:38.963214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:38.963214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:38.977157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:38.978836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:38.990828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 978/1965] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:39.143903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:39.145443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:39.158717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f98c0-b0bd-44ca-8d54-9acd0fdc180f None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:39.163717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:39.164453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:39.164682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:39.164954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] PATH_INFO: `/v3/users/601207a5abf544d9b99d477337f14026` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:39.166379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601207a5abf544d9b99d477337f14026)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:39.170893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:39.489963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:39.490786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601207a5abf544d9b99d477337f14026'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.491432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:39.491543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601207a5abf544d9b99d477337f14026'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.506017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601207a5abf544d9b99d477337f14026'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.514755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ae06-44fa-473f-a39a-f8afc94c2636 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:39.515359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601207a5abf544d9b99d477337f14026'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.536135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:39.537157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:39.537640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:39.538114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] PATH_INFO: `/v3/projects/8a85bff1226645af8921164ad515a506` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:39.540168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a85bff1226645af8921164ad515a506)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:39.542717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:39.605231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e0901776-6ef8-45d5-ba29-3f63ffb31ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601207a5abf544d9b99d477337f14026'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.615479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 988/1966] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:39 2022] DELETE /identity/v3/users/601207a5abf544d9b99d477337f14026 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:39.620092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:39.620602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:39.621108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:39.644935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:39.646474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:39.720148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 989/1967] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:39.730388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:39.731813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:39.794234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a85bff1226645af8921164ad515a506', 'request_id': 'req-47c53399-eab5-4144-83d8-08073a4ec813'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.802667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a85bff1226645af8921164ad515a506', 'request_id': 'req-47c53399-eab5-4144-83d8-08073a4ec813'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.809608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee02e0-ff92-403b-844d-efe90edb12ff None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:39.869729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:39.875646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:39.876331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:39.876722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:39.879905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:39.946584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:39.947494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:39.958669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 990/1968] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:39.960971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a85bff1226645af8921164ad515a506', 'request_id': 'req-47c53399-eab5-4144-83d8-08073a4ec813'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:39.998073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] The token cache is being invalidate because project 8a85bff1226645af8921164ad515a506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:39.998564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.000413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.009283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 979/1969] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:39 2022] DELETE /identity/v3/projects/8a85bff1226645af8921164ad515a506 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:40.013750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:40.015202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:40.029966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a876e-3d4b-44b7-98d3-50373bedd588 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:40.035930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:40.036141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:40.036521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:40.037177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] PATH_INFO: `/v3/users/9157d6eff3724042bdd7d57e74a73c92` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:40.038378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9157d6eff3724042bdd7d57e74a73c92)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:40.040877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:40.352553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9157d6eff3724042bdd7d57e74a73c92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.355947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9157d6eff3724042bdd7d57e74a73c92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.376799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9157d6eff3724042bdd7d57e74a73c92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.385586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9157d6eff3724042bdd7d57e74a73c92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.491475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9157d6eff3724042bdd7d57e74a73c92'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.492066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:40.498938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 991/1970] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:40 2022] DELETE /identity/v3/users/9157d6eff3724042bdd7d57e74a73c92 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:40.618904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:40.620618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:40.659079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53399-eab5-4144-83d8-08073a4ec813 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:40.765649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:40.766517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:40.767229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:40.767462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] PATH_INFO: `/v3/projects/de82c6d4af9e4516991e2816ad2699d8` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:40.769303 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=de82c6d4af9e4516991e2816ad2699d8)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:40.771361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:40.989438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de82c6d4af9e4516991e2816ad2699d8', 'request_id': 'req-c4d1f20e-d3a3-486d-b831-4c16101697ba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:40.999064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de82c6d4af9e4516991e2816ad2699d8', 'request_id': 'req-c4d1f20e-d3a3-486d-b831-4c16101697ba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.171631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de82c6d4af9e4516991e2816ad2699d8', 'request_id': 'req-c4d1f20e-d3a3-486d-b831-4c16101697ba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.184639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] The token cache is being invalidate because project de82c6d4af9e4516991e2816ad2699d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:41.185186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.186251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.193001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 980/1971] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:40 2022] DELETE /identity/v3/projects/de82c6d4af9e4516991e2816ad2699d8 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:41.194022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:41.196497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:41.216895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b103-cdaf-493e-ad13-503bfeb58b49 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:41.242150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:41.244010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:41.328868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:41.329501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:41.329954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:41.330413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] PATH_INFO: `/v3/users/e95cd113e0e74ecba45b2ae440f49641` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:41.331750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95cd113e0e74ecba45b2ae440f49641)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:41.334987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:41.393901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f20e-d3a3-486d-b831-4c16101697ba admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:41.411241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:41.411932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:41.412219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:41.412497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] PATH_INFO: `/v3/projects/cfe2c527411d4eceb6036eeb890693d6` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:41.413777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe2c527411d4eceb6036eeb890693d6)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:41.415718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:41.599585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95cd113e0e74ecba45b2ae440f49641'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.600836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95cd113e0e74ecba45b2ae440f49641'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.609029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe2c527411d4eceb6036eeb890693d6', 'request_id': 'req-46b3119b-bb51-40a8-bbab-6e52b6d269ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.624598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe2c527411d4eceb6036eeb890693d6', 'request_id': 'req-46b3119b-bb51-40a8-bbab-6e52b6d269ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.630901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:41.634131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95cd113e0e74ecba45b2ae440f49641'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.642346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95cd113e0e74ecba45b2ae440f49641'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.760971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe2c527411d4eceb6036eeb890693d6', 'request_id': 'req-46b3119b-bb51-40a8-bbab-6e52b6d269ad'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.783058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] The token cache is being invalidate because project cfe2c527411d4eceb6036eeb890693d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:41.783413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.784149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46b3119b-bb51-40a8-bbab-6e52b6d269ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.786080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 981/1972] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:41 2022] DELETE /identity/v3/projects/cfe2c527411d4eceb6036eeb890693d6 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:41.790898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:41.791196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:41.791477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:41.826700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:41.827986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:41.896868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e8720f8c-64d3-4882-88d7-016a953f1ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95cd113e0e74ecba45b2ae440f49641'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:41.907526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 992/1973] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:41 2022] DELETE /identity/v3/users/e95cd113e0e74ecba45b2ae440f49641 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:41.951683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 982/1974] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:41.961199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:41.961642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:41.962062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.009064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:42.016719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:42.116228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 993/1975] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:42.120583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.132106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e7b9f-b623-4f5f-a691-69a990bf74db None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.136937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.137552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.137828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.138097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.138743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:42.139043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.140693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:42.154896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 983/1976] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:42.162555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.175005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42029af9-f15a-44d0-ab00-a318966e014c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.180292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.180933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.181477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.181761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.182411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.184512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:42.496510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 994/1977] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:42 2022] POST /identity/v3/projects => generated 393 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:42.500804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.510893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028cc98-84a7-415a-b1d8-b5b4a2960471 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.570597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.571219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.571500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.571774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.572478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.574427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:42.581330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:42.743824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 984/1978] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:42 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:42.771516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.771516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6db911-fe66-4a34-851c-0602ea0991f7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.771516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.771516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.772239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.772239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.772239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.772239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:42.779458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 995/1979] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:42.781502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.792592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ff5d-be71-43d6-966f-53481ce2337a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.798503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.799218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.799588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.800005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.800760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.802887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:42.816716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 985/1980] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:42.820856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:42.831588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831a82-2ff1-4c3c-94bd-eca3dff14800 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:42.837048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:42.837736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:42.838119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:42.838319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] PATH_INFO: `/v3/projects/ad00be2c545a4f878effaf79bf3454a4/users/ced720ee888c4d2f855a4e61cddfbdba/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:42.854622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad00be2c545a4f878effaf79bf3454a4, user_id=ced720ee888c4d2f855a4e61cddfbdba, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:42.857806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d385f-3bd4-4ff5-969d-66ba3b880cda admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.087166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.089613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:43.105418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e436-0794-4d5d-bee9-0a4e433ad40f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.110754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.111415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.111627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.111994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] PATH_INFO: `/v3/projects/a6e50f1a125f4639ae0fc8e76153a826` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.113096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e50f1a125f4639ae0fc8e76153a826)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.115231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.322256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 996/1981] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:42 2022] PUT /identity/v3/projects/ad00be2c545a4f878effaf79bf3454a4/users/ced720ee888c4d2f855a4e61cddfbdba/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:43.332615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.332901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.333177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.360965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] MFA Rules not processed for user `ced720ee888c4d2f855a4e61cddfbdba`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:43.362705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:43.428094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 997/1982] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:43.432652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.499463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585d90e-7133-414c-994a-70ebf42d41bc None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.505064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.505785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.505996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.506264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.506900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.508975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.558652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e50f1a125f4639ae0fc8e76153a826', 'request_id': 'req-61918acb-596f-4a1d-806c-3f0491b4ff19'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:43.567370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e50f1a125f4639ae0fc8e76153a826', 'request_id': 'req-61918acb-596f-4a1d-806c-3f0491b4ff19'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:43.632532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 998/1983] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/projects => generated 393 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:43.635611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.672848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e50f1a125f4639ae0fc8e76153a826', 'request_id': 'req-61918acb-596f-4a1d-806c-3f0491b4ff19'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:43.691960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] The token cache is being invalidate because project a6e50f1a125f4639ae0fc8e76153a826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:43.692355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:43.693822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-61918acb-596f-4a1d-806c-3f0491b4ff19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:43.696083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 986/1984] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:43 2022] DELETE /identity/v3/projects/a6e50f1a125f4639ae0fc8e76153a826 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:43.702675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36771-d67c-4eca-bc59-127ce0b8917c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.703117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36771-d67c-4eca-bc59-127ce0b8917c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.703516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36771-d67c-4eca-bc59-127ce0b8917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.716372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed13de-d424-4dfc-9e46-89c5872e3949 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.730205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.730690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.730968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.731240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.731941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.734053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.743960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b9e36771-d67c-4eca-bc59-127ce0b8917c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:43.746156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b9e36771-d67c-4eca-bc59-127ce0b8917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:43.757482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 987/1985] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:43.768236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.772317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.772682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.812794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:43.814985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:43.828246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 988/1986] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:43.829304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.841338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de47d02-8052-4744-bf52-93a3d480781d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.848649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.849437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.860724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.860724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.860724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:43.860724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.860724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.866658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 989/1987] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:43.871464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.885481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 999/1988] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/users => generated 303 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:43.885917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46333d8-8f76-463f-ab98-f4cfb014d889 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.895567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.897871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:43.898674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.900481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.900879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.901326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.902085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.904337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.913500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e3ed-c134-43ea-a632-d2d575b5c3b8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.919007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.921183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.921183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.921183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.921183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.922661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:43.934903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1000/1989] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:43.938981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:43.949030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4b304-3226-44da-8050-4abf7a89ca22 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:43.954003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:43.954644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:43.954958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:43.955561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] PATH_INFO: `/v3/projects/30df1283573a49bf93eb55a9e3f25ba9/users/0260693467b54e6db0ec34f3c92193a4/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:43.972054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=30df1283573a49bf93eb55a9e3f25ba9, user_id=0260693467b54e6db0ec34f3c92193a4, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:43.974857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.028443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 990/1990] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:43 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:44.030709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.092840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1001/1991] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:43 2022] PUT /identity/v3/projects/30df1283573a49bf93eb55a9e3f25ba9/users/0260693467b54e6db0ec34f3c92193a4/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:44.097293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.099114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324bbd-b514-4d6e-8cd0-3305f51a8e3c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.163005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.163707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.164086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.164425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.165124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.166201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edc7df-b23a-4e96-91d9-457219d343fb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.167413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.171366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.172073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.172420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.172785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.173545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.175310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.186278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1002/1992] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:44.192230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.202987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45b9f-7e64-4736-b85c-4c942ea83037 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.209085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.209789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.210068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.210368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] PATH_INFO: `/v3/projects/30df1283573a49bf93eb55a9e3f25ba9/users/0260693467b54e6db0ec34f3c92193a4/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.212321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30df1283573a49bf93eb55a9e3f25ba9, user_id=0260693467b54e6db0ec34f3c92193a4, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.219106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad3062-6283-4d8d-99ee-fdbdbe6acc87 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.227288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1003/1993] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:44 2022] PUT /identity/v3/projects/30df1283573a49bf93eb55a9e3f25ba9/users/0260693467b54e6db0ec34f3c92193a4/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:44.238092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.238320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.238595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.268516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] MFA Rules not processed for user `0260693467b54e6db0ec34f3c92193a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:44.269745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.279065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 991/1994] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:44 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:44.283864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.293765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3900-dc88-4797-ac54-a3617e155dbd admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.351542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1004/1995] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:44.367390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.367390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.367390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.367390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.367775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.369222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.379807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 992/1996] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:44.384083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.396470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4f324-488a-4a71-b046-18057d4f1f0b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.408522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.409238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.409465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.409739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.410386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.412099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.426617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1005/1997] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:44.429482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.429642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.431519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.440465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6224e2-16cb-456a-9541-0d641d86eb47 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.445804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.446579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.446854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.447257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] PATH_INFO: `/v3/projects/0916502c9bec4047ae0a92b5fdfed7fe/users/fea6987e86e4486d9f87ca6ea13a2621/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.464250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0916502c9bec4047ae0a92b5fdfed7fe, user_id=fea6987e86e4486d9f87ca6ea13a2621, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.467109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.493299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550486-7920-4cfa-b99f-821b407d275d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.498965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.499437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.499714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.500021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.510437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.512735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972293-3b7d-41c4-ab8d-f583001c9311 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.522162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1006/1998] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:44.575629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 993/1999] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:44 2022] PUT /identity/v3/projects/0916502c9bec4047ae0a92b5fdfed7fe/users/fea6987e86e4486d9f87ca6ea13a2621/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:44.577286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.577572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.577865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.611706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] MFA Rules not processed for user `fea6987e86e4486d9f87ca6ea13a2621`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:44.612529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.618940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.650629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:44.683094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dec09-d554-42a5-a3a8-ed281f2b5132 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.692003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.692003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.692003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.692003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.692741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1007/2000] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:44.692741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.693680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.694036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.749991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:44.753906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.755351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.764964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc113178-7544-4298-bc0e-77e1f30fe674 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.767103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 994/2001] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:44.768435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.771151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.772011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.773048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.773493 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] PATH_INFO: `/v3/users/62b70f76c2f04e8684c5aa5dd3787d80` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.774798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b70f76c2f04e8684c5aa5dd3787d80)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.777735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.782134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf31060-a856-4268-ad20-35dc4506379a service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.790650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.791291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.792243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.792605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.793256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.795984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.992048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 995/2002] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:44 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:44.996782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.076120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07900-ff8d-4a55-993d-e953a3fd49c3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.081264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.081865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.082144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.082413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.083038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.085021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.129280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b70f76c2f04e8684c5aa5dd3787d80'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:45.129929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b70f76c2f04e8684c5aa5dd3787d80'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:45.144227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b70f76c2f04e8684c5aa5dd3787d80'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:45.151131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b70f76c2f04e8684c5aa5dd3787d80'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:45.219656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 996/2003] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:44 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:45.221955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.232590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd9f8-5cd6-45b7-b8ad-1a17288c9d9c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.240909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.241586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.241946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.242279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.242979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.244655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.256686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 997/2004] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:45.263749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.273613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4056b-595b-4365-8b98-900b2cbb8497 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.278385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.279026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.279336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.279602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] PATH_INFO: `/v3/projects/d906a1e33ec04eccb7979c20597b1c0a/users/3729db41a71641af9a47554a4beec611/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.295712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-35c7f27b-79f9-4b6d-bc68-da6dc84fac63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b70f76c2f04e8684c5aa5dd3787d80'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:45.296078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d906a1e33ec04eccb7979c20597b1c0a, user_id=3729db41a71641af9a47554a4beec611, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.298846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.310535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1008/2005] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:44 2022] DELETE /identity/v3/users/62b70f76c2f04e8684c5aa5dd3787d80 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:45.315004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.315283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.315556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.335946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:45.337186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.408604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1009/2006] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:45.412879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.414415 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.476785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edce578-4f7f-4481-aab0-f2219e961998 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.485033 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 998/2007] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:45 2022] PUT /identity/v3/projects/d906a1e33ec04eccb7979c20597b1c0a/users/3729db41a71641af9a47554a4beec611/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:45.485997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.488504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.532022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.532641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.532914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.533737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.561092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f971-7c19-45ec-a403-d350b4308c2f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.566222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.567140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.567370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.567747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.572865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.572865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.581720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 999/2008] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:45.585973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.596152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e6bc8-1d11-4719-87cd-dcb5215638f7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.601803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.602471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.602771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.603045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] PATH_INFO: `/v3/projects/d906a1e33ec04eccb7979c20597b1c0a/users/3729db41a71641af9a47554a4beec611/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.605072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d906a1e33ec04eccb7979c20597b1c0a, user_id=3729db41a71641af9a47554a4beec611, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.606738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.608346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.608500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6033033-79e7-4c32-951f-f4889df5547f service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.619502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1010/2009] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:45.621395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1000/2010] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:45 2022] PUT /identity/v3/projects/d906a1e33ec04eccb7979c20597b1c0a/users/3729db41a71641af9a47554a4beec611/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:45.632075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.632497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.632692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.667037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] MFA Rules not processed for user `3729db41a71641af9a47554a4beec611`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:45.669151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.773589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1011/2011] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:45.796644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:45.798694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.860053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b9448-66c1-4015-8f7f-270381e39575 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:45.863863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:45.864524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:45.864814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:45.865065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:45.866947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:45.939214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:45.941409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:45.953115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1001/2012] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:46.125495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:46.127579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:46.231016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e123e-a726-4f05-9702-e902bfec5939 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:46.238486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:46.239008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:46.239289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:46.239564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] PATH_INFO: `/v3/projects/9b4d5ce638d74ba88d0d2c8cff37c3e0` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:46.240746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4d5ce638d74ba88d0d2c8cff37c3e0)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:46.242915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:46.274734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:46.377510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4d5ce638d74ba88d0d2c8cff37c3e0', 'request_id': 'req-c490f850-52fc-4b87-a29f-aaf0b509c7e0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.383196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4d5ce638d74ba88d0d2c8cff37c3e0', 'request_id': 'req-c490f850-52fc-4b87-a29f-aaf0b509c7e0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.391246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4d5ce638d74ba88d0d2c8cff37c3e0', 'request_id': 'req-c490f850-52fc-4b87-a29f-aaf0b509c7e0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.490206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] The token cache is being invalidate because project 9b4d5ce638d74ba88d0d2c8cff37c3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:46.490578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.491296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.493196 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1012/2013] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:46 2022] DELETE /identity/v3/projects/9b4d5ce638d74ba88d0d2c8cff37c3e0 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:46.497226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:46.508581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:46.512734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3855-3a0c-441f-8d85-f0df0cb59261 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:46.579697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:46.580533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:46.580807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:46.581078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] PATH_INFO: `/v3/users/a4ea08794d6042d0bc63043c441b8e6b` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:46.582339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ea08794d6042d0bc63043c441b8e6b)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:46.584455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:46.847766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ea08794d6042d0bc63043c441b8e6b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.975507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ea08794d6042d0bc63043c441b8e6b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.976244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ea08794d6042d0bc63043c441b8e6b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:46.994304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ea08794d6042d0bc63043c441b8e6b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:47.010106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ea08794d6042d0bc63043c441b8e6b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:47.016887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1002/2014] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:46 2022] DELETE /identity/v3/users/a4ea08794d6042d0bc63043c441b8e6b => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:47.627985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:47.629837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:47.651670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490f850-52fc-4b87-a29f-aaf0b509c7e0 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:47.724885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:47.725538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:47.725812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:47.726081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] PATH_INFO: `/v3/projects/5723b400d8f544e08688ddba95002f4f` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:47.727251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5723b400d8f544e08688ddba95002f4f)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:47.729277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:47.924563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5723b400d8f544e08688ddba95002f4f', 'request_id': 'req-7579454e-1cb9-400c-a4d2-00de2b029100'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:47.929915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5723b400d8f544e08688ddba95002f4f', 'request_id': 'req-7579454e-1cb9-400c-a4d2-00de2b029100'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:47.938640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5723b400d8f544e08688ddba95002f4f', 'request_id': 'req-7579454e-1cb9-400c-a4d2-00de2b029100'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.128718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] The token cache is being invalidate because project 5723b400d8f544e08688ddba95002f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:48.129455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.130297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.132439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1013/2015] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:47 2022] DELETE /identity/v3/projects/5723b400d8f544e08688ddba95002f4f => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:48.137552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:48.141431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:48.171030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba720-f59d-4610-abc1-a17e85f1eb6c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:48.220150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:48.245230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:48.245230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:48.245230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:48.245230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] PATH_INFO: `/v3/users/357ca573cf4b45939d5665a2732ff63d` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:48.245742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=357ca573cf4b45939d5665a2732ff63d)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:48.247371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:48.478151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357ca573cf4b45939d5665a2732ff63d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.629864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357ca573cf4b45939d5665a2732ff63d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.630632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357ca573cf4b45939d5665a2732ff63d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.640750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357ca573cf4b45939d5665a2732ff63d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.669389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ca0fbbe1-1be6-4d75-bf5b-e7e102e0b23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357ca573cf4b45939d5665a2732ff63d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:48.681749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1003/2016] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:48 2022] DELETE /identity/v3/users/357ca573cf4b45939d5665a2732ff63d => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:49.248026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:49.260250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:49.278677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579454e-1cb9-400c-a4d2-00de2b029100 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:49.335681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:49.336408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:49.336691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:49.336976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] PATH_INFO: `/v3/projects/8e6f56acce0c4735838ce64b4c9ee051` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:49.338136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6f56acce0c4735838ce64b4c9ee051)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:49.340506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:49.343932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:49.485717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6f56acce0c4735838ce64b4c9ee051', 'request_id': 'req-040628b5-4330-408c-b435-64e76a59ab60'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:49.491633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6f56acce0c4735838ce64b4c9ee051', 'request_id': 'req-040628b5-4330-408c-b435-64e76a59ab60'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:49.498543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6f56acce0c4735838ce64b4c9ee051', 'request_id': 'req-040628b5-4330-408c-b435-64e76a59ab60'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:49.582409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] The token cache is being invalidate because project 8e6f56acce0c4735838ce64b4c9ee051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:49.582626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:49.583207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-040628b5-4330-408c-b435-64e76a59ab60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:49.584745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1014/2017] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:49 2022] DELETE /identity/v3/projects/8e6f56acce0c4735838ce64b4c9ee051 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:49.591831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:49.596329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:49.596329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:49.613596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:49.614814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:49.694097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1004/2018] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:49.703661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:49.704081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:49.704442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:49.728661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:49.730070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:49.741726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1015/2019] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:49.747311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:49.758954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98938f-6ec9-4782-97a6-918de8e8b2f5 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:49.764412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:49.765159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:49.765557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:49.765977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:49.766837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:49.767223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:49.769023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:49.786764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1005/2020] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:49.790999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:49.801484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7b1bb-981b-42ae-acea-5cc742937705 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:49.807425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:49.808178 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:49.808517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:49.808850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:49.809562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:49.811518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:49.912539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1016/2021] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:49 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:49.917664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:49.934850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae264e-0b32-4d90-b7d0-aef8ec26eb0b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.016499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.018229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.018829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.019164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.019818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.022198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.226878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1006/2022] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:49 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:50.231874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.243128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b9883-4950-48d3-8207-c72577c5c4c6 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.247928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.248570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.251941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.251941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.251941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.251941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.263927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1017/2023] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:50.267088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.268649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.270354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.277572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519a16a-457f-4fad-bdea-b6928e8d17a3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.284406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2539-7531-4dd2-960d-79ed913af05a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.288685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.289333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.289609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.289730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.289873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.290250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.290529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.290984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] PATH_INFO: `/v3/users/7afbfa44402646d4a9d3e8c5cb104214` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.291136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.292615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afbfa44402646d4a9d3e8c5cb104214)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.292747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.294547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.306424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1007/2024] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:50.310676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.323005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9e70-8c4e-4089-a52b-5ca80ae50987 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.327869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.328565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.328847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.329117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] PATH_INFO: `/v3/projects/354e8e93d274406191fa2737dce771ec/users/798c86aec9ee4154871225dddf50f554/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.344067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354e8e93d274406191fa2737dce771ec, user_id=798c86aec9ee4154871225dddf50f554, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.346971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e69bfb-2f77-49dc-9a3a-15d25a29f462 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.443175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1008/2025] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:50 2022] PUT /identity/v3/projects/354e8e93d274406191fa2737dce771ec/users/798c86aec9ee4154871225dddf50f554/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:50.460703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.461013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.468102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.494970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] MFA Rules not processed for user `798c86aec9ee4154871225dddf50f554`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:50.496257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.555998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afbfa44402646d4a9d3e8c5cb104214'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.557047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afbfa44402646d4a9d3e8c5cb104214'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.562122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1009/2026] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:50.571517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afbfa44402646d4a9d3e8c5cb104214'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.573906 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.575307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.578728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afbfa44402646d4a9d3e8c5cb104214'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.632448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a411c-c6f6-4e17-9286-8198cdddc4ea None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.636208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.639495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.639495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.639495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.646174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.647610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5640489-a710-433d-81a4-9037b7fb98bf service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.656713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1010/2027] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:50.657109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afbfa44402646d4a9d3e8c5cb104214'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.665184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1018/2028] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:50 2022] DELETE /identity/v3/users/7afbfa44402646d4a9d3e8c5cb104214 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:50.669754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.670041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.670316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.682619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.683982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.690216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:50.691401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.748230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:50.749568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe3912-45b7-41d5-948e-26a4de777f46 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.755982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.756813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1011/2029] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:50.756994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.757286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.757561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] PATH_INFO: `/v3/users/798c86aec9ee4154871225dddf50f554` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.758803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=798c86aec9ee4154871225dddf50f554)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.760945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.772877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:50.774388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:50.788292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396763-9e90-46c3-b07f-5d7461d55620 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:50.793557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:50.794240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:50.794755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:50.795031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] PATH_INFO: `/v3/users/ced720ee888c4d2f855a4e61cddfbdba` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:50.796214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced720ee888c4d2f855a4e61cddfbdba)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:50.798537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:50.969893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798c86aec9ee4154871225dddf50f554'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.971375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798c86aec9ee4154871225dddf50f554'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.984015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798c86aec9ee4154871225dddf50f554'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:50.991703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798c86aec9ee4154871225dddf50f554'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.010125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced720ee888c4d2f855a4e61cddfbdba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.102564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798c86aec9ee4154871225dddf50f554'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.109217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1019/2030] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:50 2022] DELETE /identity/v3/users/798c86aec9ee4154871225dddf50f554 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:51.111469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:51.112866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.153928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced720ee888c4d2f855a4e61cddfbdba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.154743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced720ee888c4d2f855a4e61cddfbdba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.162689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced720ee888c4d2f855a4e61cddfbdba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.170010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900f18d-c4ce-49a1-b147-5b34d95f5735 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:51.173753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:51.174361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.174667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.174931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.175778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-850cfbce-d287-4d8e-b2e0-71bf6b76a275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced720ee888c4d2f855a4e61cddfbdba'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:51.176704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.181883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1012/2031] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:50 2022] DELETE /identity/v3/users/ced720ee888c4d2f855a4e61cddfbdba => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:51.184889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.184889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.184889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.203388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:51.204556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.246323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:51.247974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc0fe8-c825-42a3-a372-27ea4702f433 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:51.258965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1020/2032] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:51.260537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.261401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.261777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.291051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:51.292290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.299191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1013/2033] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:51.310922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:51.312344 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1021/2034] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:51.316201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.324140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:51.325697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.382715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e4a91-0c81-4513-b835-e101b469758c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:51.388255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:51.388923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.389319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.389602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.399489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d1eda-afb3-4118-aa6a-385d9933571d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:51.400161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:51.401605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:51.403408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:51.404168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.404445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.404714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.411142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1014/2035] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:51.414846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:51.416499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:51.426525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1022/2036] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:51.478962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:51.898758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:51.900200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:51.900381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.903311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:51.914225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead60c2e-472c-4f4d-b5fd-8b58a1b9e95a service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:51.919074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:51.919740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.920024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.920284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] PATH_INFO: `/v3/projects/c8136e1fd1e14383be9330627b0f1111` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.921799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8136e1fd1e14383be9330627b0f1111)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:51.924114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:51.930172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91260609-24d5-49ec-9ad5-7e1fae1d0686 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:51.940609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:51.941364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:51.941758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:51.942040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] PATH_INFO: `/v3/projects/354e8e93d274406191fa2737dce771ec` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:51.943644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354e8e93d274406191fa2737dce771ec)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:51.945717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.094154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354e8e93d274406191fa2737dce771ec', 'request_id': 'req-5106e4c8-41b6-4945-a021-a7af1b24ba50'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.095244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8136e1fd1e14383be9330627b0f1111', 'request_id': 'req-07351015-8544-47fe-8e35-81c1c9b47a41'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.100502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354e8e93d274406191fa2737dce771ec', 'request_id': 'req-5106e4c8-41b6-4945-a021-a7af1b24ba50'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.103550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8136e1fd1e14383be9330627b0f1111', 'request_id': 'req-07351015-8544-47fe-8e35-81c1c9b47a41'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.107135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354e8e93d274406191fa2737dce771ec', 'request_id': 'req-5106e4c8-41b6-4945-a021-a7af1b24ba50'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.184412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8136e1fd1e14383be9330627b0f1111', 'request_id': 'req-07351015-8544-47fe-8e35-81c1c9b47a41'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.198544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] The token cache is being invalidate because project c8136e1fd1e14383be9330627b0f1111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:52.198840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.199690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.206422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1015/2037] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:51 2022] DELETE /identity/v3/projects/c8136e1fd1e14383be9330627b0f1111 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:52.207639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.209087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.274206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] The token cache is being invalidate because project 354e8e93d274406191fa2737dce771ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:52.274778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.275337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5106e4c8-41b6-4945-a021-a7af1b24ba50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.276930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1023/2038] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:51 2022] DELETE /identity/v3/projects/354e8e93d274406191fa2737dce771ec => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:52.278674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c4ba-bf6d-49c9-ae7e-50f02f9e8c57 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.279113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c4ba-bf6d-49c9-ae7e-50f02f9e8c57 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.279510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c4ba-bf6d-49c9-ae7e-50f02f9e8c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.285950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07351015-8544-47fe-8e35-81c1c9b47a41 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.306418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5a90c4ba-bf6d-49c9-ae7e-50f02f9e8c57 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:52.307910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5a90c4ba-bf6d-49c9-ae7e-50f02f9e8c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.355507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.356185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.356467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.357514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] PATH_INFO: `/v3/projects/ad00be2c545a4f878effaf79bf3454a4` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.359928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad00be2c545a4f878effaf79bf3454a4)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.363003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.379216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1024/2039] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:52.385303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444efd1-08b2-4c89-9f00-4a1a662518cb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.385788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444efd1-08b2-4c89-9f00-4a1a662518cb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.386067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444efd1-08b2-4c89-9f00-4a1a662518cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.405747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d444efd1-08b2-4c89-9f00-4a1a662518cb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:52.406941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d444efd1-08b2-4c89-9f00-4a1a662518cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.414915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1025/2040] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:52.416298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a47822-6f4c-4ca8-852c-36fdcded59d4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.416549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a47822-6f4c-4ca8-852c-36fdcded59d4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.416819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a47822-6f4c-4ca8-852c-36fdcded59d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.436874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-27a47822-6f4c-4ca8-852c-36fdcded59d4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:52.438048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-27a47822-6f4c-4ca8-852c-36fdcded59d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.446559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1026/2041] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:52.447874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.448163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.448432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.467470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:52.468771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.477009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1027/2042] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:52.477815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.488134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8ecc2-b093-41a0-8284-f0c0eca8730b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.492762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.493352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.493629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.494042 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.494687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:52.495157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.496678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.510325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1028/2043] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:52.511128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.515883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad00be2c545a4f878effaf79bf3454a4', 'request_id': 'req-f0f0f712-106c-4955-94fe-d08b747b3c44'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.521443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f496a83-177e-4d97-bf91-7083728dfc4b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.523102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad00be2c545a4f878effaf79bf3454a4', 'request_id': 'req-f0f0f712-106c-4955-94fe-d08b747b3c44'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.585404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.586018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.586297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.586569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.587341 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:52.587645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.589315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.601487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1029/2044] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:52.602403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.613369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8cac-41ff-4815-acb8-14a0cc004e95 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.624928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.625547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.625826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.626093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.627886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.630179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.643943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad00be2c545a4f878effaf79bf3454a4', 'request_id': 'req-f0f0f712-106c-4955-94fe-d08b747b3c44'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.656060 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] The token cache is being invalidate because project ad00be2c545a4f878effaf79bf3454a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:52.656351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.657094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:52.659179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1016/2045] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:52 2022] DELETE /identity/v3/projects/ad00be2c545a4f878effaf79bf3454a4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:52.659964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.661285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.674696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0f712-106c-4955-94fe-d08b747b3c44 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.679539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.682514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.682514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.682514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.682514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.683317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.745236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1030/2046] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:52.745236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.748830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.816407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf528-8bfd-4eed-80d3-92c2e90a2852 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.821386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.822140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.823078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.823481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] PATH_INFO: `/v3/users/0260693467b54e6db0ec34f3c92193a4` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.825168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0260693467b54e6db0ec34f3c92193a4)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.827086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:52.842493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1017/2047] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:52.842493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:52.842493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:52.920844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5923-3ab5-4aa8-8e88-930126e6adfc admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:52.932690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:52.933324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:52.933598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:52.933872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:52.934511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:52.937223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.065321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1018/2048] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:52 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:53.066781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.074307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0260693467b54e6db0ec34f3c92193a4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:53.075923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0260693467b54e6db0ec34f3c92193a4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:53.080246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4303-2275-4b77-922e-57e6b981064f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.085862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.086939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.087371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.088014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.088819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.090044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0260693467b54e6db0ec34f3c92193a4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:53.091362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.097504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0260693467b54e6db0ec34f3c92193a4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:53.125471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:53.195704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0260693467b54e6db0ec34f3c92193a4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:53.202323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1019/2049] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:53 2022] POST /identity/v3/users => generated 281 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:53.205936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1031/2050] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:52 2022] DELETE /identity/v3/users/0260693467b54e6db0ec34f3c92193a4 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:53.206342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.207309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.209085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.209322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.222141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184f90-df51-466f-a368-9087de2b5645 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.285943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1932-421d-460a-bd84-76ca5c0132fd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.291333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.292288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.292736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.293153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.293948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.295904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.310292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1032/2051] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:53.311412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.312078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.312464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.312960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.313494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.314149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.319523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.326459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322787c-7646-4d27-8905-e3104035c39b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.334905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1020/2052] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:53.337570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.338369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.338533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.338756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.339035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.339694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.343747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.357649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001593d1-7a14-4423-a1d0-fb1d246dcf76 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.360570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1033/2053] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:53.362211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.365393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.366062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.366341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.366619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.367315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.368969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.373552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12841a-4e01-4dba-8f0b-c7faf553a01d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.380316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.381160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.381443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.382006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] PATH_INFO: `/v3/projects/28356b4151bc404480cf7ad77a13bb4f/users/213e1b4712a54dfebf39edf8f75ede46/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.399235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1021/2054] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:53.399235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.404031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] RBAC: Authorizing `identity:create_grant(project_id=28356b4151bc404480cf7ad77a13bb4f, user_id=213e1b4712a54dfebf39edf8f75ede46, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.404222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50775d-7ee9-4842-a626-1931b0d3257b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.406913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4604b-2bdb-4df0-9ff9-6150749f002b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.412600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.413004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.413357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.413562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] PATH_INFO: `/v3/projects/d5674263f170421281aa132825e852a9/users/c1ed0c9d7fe3460bbc1d00815f9e95bd/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.432866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5674263f170421281aa132825e852a9, user_id=c1ed0c9d7fe3460bbc1d00815f9e95bd, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.433684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bb81d-1e2b-4714-ad3d-7450a0256610 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.521566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1034/2055] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:53 2022] PUT /identity/v3/projects/28356b4151bc404480cf7ad77a13bb4f/users/213e1b4712a54dfebf39edf8f75ede46/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:53.537122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.537345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.537618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.566266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1022/2056] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:53 2022] PUT /identity/v3/projects/d5674263f170421281aa132825e852a9/users/c1ed0c9d7fe3460bbc1d00815f9e95bd/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:53.567904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.568193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.568472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.570066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] MFA Rules not processed for user `213e1b4712a54dfebf39edf8f75ede46`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:53.571434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.600390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] MFA Rules not processed for user `c1ed0c9d7fe3460bbc1d00815f9e95bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:53.601656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.640842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1035/2057] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:53.669621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1023/2058] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:53.674912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.681681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.683151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.746417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd672-df94-4e02-9b1b-a0d72b7d55de None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.747927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b57e-5ecb-47de-8820-12ed916fbae1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.751799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.752069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.752582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.752704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.752872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.752967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.753191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.753803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.753938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.756008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.764639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.766117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.775973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1024/2059] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:53.777010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.778145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:53.793025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c224-0530-4c95-996e-102b08c82fad service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.797718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.798467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.798744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.799016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] PATH_INFO: `/v3/projects/30df1283573a49bf93eb55a9e3f25ba9` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.800260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30df1283573a49bf93eb55a9e3f25ba9)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.802167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:53.912956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1036/2060] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:53 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:53.912956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:53.975354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32bbe-706e-4e4d-a47a-becb3bde5cc9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:53.979828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:53.980494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:53.980780 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:53.981047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] PATH_INFO: `/v3/users/213e1b4712a54dfebf39edf8f75ede46` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:53.982418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213e1b4712a54dfebf39edf8f75ede46)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:53.984744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.004965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30df1283573a49bf93eb55a9e3f25ba9', 'request_id': 'req-5fa65a22-f1ad-4130-82f6-af816e609cd7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.012178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30df1283573a49bf93eb55a9e3f25ba9', 'request_id': 'req-5fa65a22-f1ad-4130-82f6-af816e609cd7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.130977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30df1283573a49bf93eb55a9e3f25ba9', 'request_id': 'req-5fa65a22-f1ad-4130-82f6-af816e609cd7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.143887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] The token cache is being invalidate because project 30df1283573a49bf93eb55a9e3f25ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:54.144173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.144833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.146336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1025/2061] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:53 2022] DELETE /identity/v3/projects/30df1283573a49bf93eb55a9e3f25ba9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:54.147141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.148547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.211969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65a22-f1ad-4130-82f6-af816e609cd7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.216784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.217310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.217587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.217859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.218597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.220673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.265349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213e1b4712a54dfebf39edf8f75ede46'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.265996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213e1b4712a54dfebf39edf8f75ede46'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.278352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213e1b4712a54dfebf39edf8f75ede46'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.285043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213e1b4712a54dfebf39edf8f75ede46'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.319724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1026/2062] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:54.320523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.322100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.342753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:54.382185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1936272-4a25-4fd2-9171-ec4c8fe74f35 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.386280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.386886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.387178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.387447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.389181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.411085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8ca7cc9a-042e-4823-9317-06b1edbff630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213e1b4712a54dfebf39edf8f75ede46'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:54.418680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1037/2063] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:53 2022] DELETE /identity/v3/users/213e1b4712a54dfebf39edf8f75ede46 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:54.420124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ffab-6d41-40b0-b3f4-98535773f486 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.420414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ffab-6d41-40b0-b3f4-98535773f486 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.420741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ffab-6d41-40b0-b3f4-98535773f486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.442557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-fa48ffab-6d41-40b0-b3f4-98535773f486 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:54.443796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fa48ffab-6d41-40b0-b3f4-98535773f486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.465244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.467228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.480825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1027/2064] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:54.482246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.495830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e52722-80b8-40ed-8347-8533e1b30b67 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.534339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1038/2065] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:54.548943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.549306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.549646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.568085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.573031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.573031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.573031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.573031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.573031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce6d9e-67cd-4ac8-82c5-2923da15fdd4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.587768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1028/2066] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:54.589239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.589523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.589799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.604836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:54.606079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.619904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1039/2067] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:54.619904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.619904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.642049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:54.643023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce05930-f340-4fb0-97ae-8c6227fce9af None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.643271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.654290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.655082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.655461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.655894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] PATH_INFO: `/v3/projects/0b7063789ac94ea999602de1205e56fd/users/febdf476ca954e3dbbf228c1e28b5f55/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.659832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1029/2068] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:54.664020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.666991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.674312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7063789ac94ea999602de1205e56fd, user_id=febdf476ca954e3dbbf228c1e28b5f55, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.677715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.761462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001ceeb-7438-477b-be92-06b0b6881264 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.765481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.766097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.766371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.766637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.777146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1040/2069] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:54 2022] PUT /identity/v3/projects/0b7063789ac94ea999602de1205e56fd/users/febdf476ca954e3dbbf228c1e28b5f55/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:54.778175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.781914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.783532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.793987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1030/2070] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:54.794852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.804726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a5bc-d05c-4013-b32f-10d9cdddced5 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.857853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be057a-e956-4236-87ab-a3df6e37993d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.865172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.865934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.866220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.866492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.867266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.867396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:54.867674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.867964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.868244 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.868515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.869429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.869640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.871255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.883826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1031/2071] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:54.884550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1041/2072] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:54.889405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.889556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:54.899209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc74-0f12-4353-b15f-cb2c99ead39c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.904609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.905883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.906155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.906850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] PATH_INFO: `/v3/projects/0b7063789ac94ea999602de1205e56fd/users/febdf476ca954e3dbbf228c1e28b5f55/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.911076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7063789ac94ea999602de1205e56fd, user_id=febdf476ca954e3dbbf228c1e28b5f55, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.914316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952df7d-7fd6-4051-a3d2-56c1de7131d7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:54.921802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43451a-fceb-4166-b48a-924948a0e293 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:54.926946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1032/2073] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:54 2022] PUT /identity/v3/projects/0b7063789ac94ea999602de1205e56fd/users/febdf476ca954e3dbbf228c1e28b5f55/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:54.937256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.937614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.937950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.966593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] MFA Rules not processed for user `febdf476ca954e3dbbf228c1e28b5f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:54.968380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:54.987769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:54.988409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:54.988684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:54.988955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:54.989579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:54.991736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.037573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1033/2074] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:55.052320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.053761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.123600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca80de-4e87-48c6-b392-a5903180c1ed None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.123600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.124205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.124481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.124753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.126628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.150353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1042/2075] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:54 2022] POST /identity/v3/projects => generated 383 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:55.156074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.193743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.195201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.204680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1034/2076] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:55.237128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b808fc8-c3a6-4935-bd92-8e0c18be8bbf admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.242821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.243401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.243786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.244048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.244671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.246589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.317482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.319384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.338161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47747267-f4a6-4ee3-8ed7-b159c27fc115 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.351390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.351390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.351390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.351390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] PATH_INFO: `/v3/projects/28356b4151bc404480cf7ad77a13bb4f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.352163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28356b4151bc404480cf7ad77a13bb4f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.352163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.387984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1043/2077] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:55 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:55.390988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.401778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198b559-c73d-4d6c-b67c-4356bcb9e9ad admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.406460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.407057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.407335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.407611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.408997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.410592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.421901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1044/2078] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:55.431274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.441434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbbf4-b2f8-4106-b01b-7a35ef2d6446 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.446357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.447070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.447347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.447746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.448423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.450049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.465418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1045/2079] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:55.467474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.478862 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568a70-badc-446f-af74-bd7e5d540ce9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.483589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.484673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.484964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.485237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] PATH_INFO: `/v3/projects/8b33ad6f61234c81b5d7692f96011e44/users/28e07a26a97c46b88cc213a5e745d9a6/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.503889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b33ad6f61234c81b5d7692f96011e44, user_id=28e07a26a97c46b88cc213a5e745d9a6, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.512010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72916-cd7f-41ff-9c44-a4da5513075b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.518561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28356b4151bc404480cf7ad77a13bb4f', 'request_id': 'req-13993e98-f035-40ce-ace7-941d32aaf8e1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:55.524136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28356b4151bc404480cf7ad77a13bb4f', 'request_id': 'req-13993e98-f035-40ce-ace7-941d32aaf8e1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:55.644760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1046/2080] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:55 2022] PUT /identity/v3/projects/8b33ad6f61234c81b5d7692f96011e44/users/28e07a26a97c46b88cc213a5e745d9a6/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:55.655198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b3a33-886c-40c1-960c-0ad6dcb9c9e5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.655481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b3a33-886c-40c1-960c-0ad6dcb9c9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.655758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b3a33-886c-40c1-960c-0ad6dcb9c9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.683518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-403b3a33-886c-40c1-960c-0ad6dcb9c9e5 None None] MFA Rules not processed for user `28e07a26a97c46b88cc213a5e745d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:55.684722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-403b3a33-886c-40c1-960c-0ad6dcb9c9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.702185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28356b4151bc404480cf7ad77a13bb4f', 'request_id': 'req-13993e98-f035-40ce-ace7-941d32aaf8e1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:55.717345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] The token cache is being invalidate because project 28356b4151bc404480cf7ad77a13bb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:55.718220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:55.719316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-13993e98-f035-40ce-ace7-941d32aaf8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:55.721231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1035/2081] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:55 2022] DELETE /identity/v3/projects/28356b4151bc404480cf7ad77a13bb4f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:55.725875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.728587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.729829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.753546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:55.754903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.815954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1047/2082] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:55.850984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1036/2083] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:55.860268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.860629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.861689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.883473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:55.884829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.893316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1048/2084] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:55.898595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.912188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f4619-7739-412a-a66f-9011f4c631d4 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.917825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.918640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.919011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.919378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.920246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:55.920694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.922394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.926006 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.927433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.943912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1037/2085] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:55.945308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.956855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889752b-6af0-485c-92fd-8e5fc7ca8a34 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.963305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.975040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.975040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.975040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.975040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.975040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.999231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0302cd7-412b-4ec8-a01b-474bd7081346 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.003786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.004439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.004713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.006239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.013870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.015312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.025729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1049/2086] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:56.065666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.068232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:56.074631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:56.087156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9882-569a-40e4-a856-8b892e509a2f service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.092550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.093184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.093459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.093740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] PATH_INFO: `/v3/users/28e07a26a97c46b88cc213a5e745d9a6` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.095021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e07a26a97c46b88cc213a5e745d9a6)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.097241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.163631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1038/2087] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:55 2022] POST /identity/v3/projects => generated 373 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:56.164744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.238837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7e1be-d61d-4664-a611-7954c2b05eaa admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.244149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.244766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.245039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.245312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.245943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.248034 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.278871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:56.331916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e07a26a97c46b88cc213a5e745d9a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:56.332693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e07a26a97c46b88cc213a5e745d9a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:56.347599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e07a26a97c46b88cc213a5e745d9a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:56.362547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e07a26a97c46b88cc213a5e745d9a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:56.370582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1039/2088] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:56 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:56.375705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.397675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a574-c9a5-4ad6-aeca-e0da9a133ccd admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.403385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.404204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.404654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.404762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.405516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.407656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.431746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1040/2089] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:56.435959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.449283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cdbdc-ce1e-4084-bf23-a190f1fd7726 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.454949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.456024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.457535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.457535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.458091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.459947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.473570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1041/2090] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:56.478164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.485675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9d6831cc-69f7-49e9-bb9f-570806aece1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e07a26a97c46b88cc213a5e745d9a6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:56.493796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1050/2091] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:56 2022] DELETE /identity/v3/users/28e07a26a97c46b88cc213a5e745d9a6 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:56.499899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c8e5-0a52-4875-8d6d-6837004366a3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.502011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.502482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.502957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.547348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:56.550295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:56.572495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.573247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.573546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.573827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] PATH_INFO: `/v3/projects/7928dd93bf3044848d6469e4431f4cbd/users/d9ff5501dbfa466482af479c9b12d943/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.607151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7928dd93bf3044848d6469e4431f4cbd, user_id=d9ff5501dbfa466482af479c9b12d943, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.626272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00ad5e-1a4a-45a2-b5cf-8cdec1a627f3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.647614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1051/2092] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:56.669653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:56.669653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:56.716311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286577a-2b56-4f32-92bb-8cf4b8778dac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:56.720631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:56.721311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.721524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.721794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.727688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1042/2093] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:56 2022] PUT /identity/v3/projects/7928dd93bf3044848d6469e4431f4cbd/users/d9ff5501dbfa466482af479c9b12d943/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:56.737295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:56.737295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:56.739696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:56.740665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:56.741078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:56.745747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1052/2094] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:56.756703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:56.803892 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] MFA Rules not processed for user `d9ff5501dbfa466482af479c9b12d943`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:56.805154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:56.942688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1043/2095] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:57.164542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:57.169630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.243702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b29318-3589-472e-9a52-2c41d4a9a4bb service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:57.323782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:57.323782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.323782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.323782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.332148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:57.333817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:57.347547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1053/2096] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:57.380576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:57.389251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.489221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147a113-cd1f-4702-9534-fa990da55643 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:57.494591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:57.495290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.495567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.495838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] PATH_INFO: `/v3/users/d9ff5501dbfa466482af479c9b12d943` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.496980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ff5501dbfa466482af479c9b12d943)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:57.498931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:57.573169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:57.575222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.589244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ef48-d7d7-4dcd-99a2-f101f56eb288 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:57.597809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:57.598703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.598880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.599000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] PATH_INFO: `/v3/projects/8b33ad6f61234c81b5d7692f96011e44` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.600714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b33ad6f61234c81b5d7692f96011e44)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:57.603637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:57.711443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ff5501dbfa466482af479c9b12d943'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.771661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b33ad6f61234c81b5d7692f96011e44', 'request_id': 'req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.777547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b33ad6f61234c81b5d7692f96011e44', 'request_id': 'req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.784907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b33ad6f61234c81b5d7692f96011e44', 'request_id': 'req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.788799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ff5501dbfa466482af479c9b12d943'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.789528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ff5501dbfa466482af479c9b12d943'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.798004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ff5501dbfa466482af479c9b12d943'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.812679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-51d4081e-2cd3-4636-b2fc-eae02f1fb8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ff5501dbfa466482af479c9b12d943'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.820314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1044/2097] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:57 2022] DELETE /identity/v3/users/d9ff5501dbfa466482af479c9b12d943 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:57.824663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.824951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.825220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.844686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:57.845904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.905223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1045/2098] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:57.915493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:57.916905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.926960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] The token cache is being invalidate because project 8b33ad6f61234c81b5d7692f96011e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:57.927247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.927921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5b7fc8f2-66bd-471f-8f04-facfe2767cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:57.930836 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1054/2099] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:57 2022] DELETE /identity/v3/projects/8b33ad6f61234c81b5d7692f96011e44 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:57.934399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65adb-49f9-40ae-a6fe-bdd835f7e4b5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.934680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65adb-49f9-40ae-a6fe-bdd835f7e4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.934954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65adb-49f9-40ae-a6fe-bdd835f7e4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.955397 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d8a65adb-49f9-40ae-a6fe-bdd835f7e4b5 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:57.956729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d8a65adb-49f9-40ae-a6fe-bdd835f7e4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.966492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1055/2100] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:57.974845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.975135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.975427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.981223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10bbe9-1f20-4dac-8d7e-67c550ec8ef3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:57.990358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:57.991185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.992205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.992620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.996050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.998060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:57.999237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.011468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.012151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:58.012978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.015641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1056/2101] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:21:58.019609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.026050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1046/2102] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:21:58.029695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db012b6-aee9-4a43-840a-014ebd283727 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.034940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.035550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.035844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.036121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.036793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:58.037163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.038825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.051886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1057/2103] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:21:58.056250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.067091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62750-a4a2-4075-8992-7a907aeb95a4 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.077410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.078030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.078304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.078727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.079624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.081704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.278960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1047/2104] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:58 2022] POST /identity/v3/projects => generated 369 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:58.282249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.294181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47fbb-fde7-414b-9452-e7bd16d116ff admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.404272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.412059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.412059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.412059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.412059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.412059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.563506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1058/2105] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:58 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:58.566400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.580668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.584122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.590905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a270e-6ba8-43be-96cf-5357a1bfa044 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.602865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5dd1a-d316-4ebf-b184-cde4e55ff93d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.605208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.606188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.606475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.606788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.607445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.609172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.613151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.613887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.614216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.614552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] PATH_INFO: `/v3/projects/7928dd93bf3044848d6469e4431f4cbd` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.615753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7928dd93bf3044848d6469e4431f4cbd)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.617757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.632566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1048/2106] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:58.636766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.646939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e8e7-1383-452b-9af8-36cf83f25d03 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.651834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.652524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.652799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.653070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.653704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.655296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.666747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1049/2107] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:21:58.670625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.681753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb67c-c1a7-4c45-a92e-714d72edbae7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.686293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.686939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.687215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.687489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] PATH_INFO: `/v3/projects/e69948abdb664038ba01c95e87fcb4eb/users/d4b43908501d4bc0951deb3463e67642/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.702357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69948abdb664038ba01c95e87fcb4eb, user_id=d4b43908501d4bc0951deb3463e67642, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.705095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.836869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1050/2108] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:58 2022] PUT /identity/v3/projects/e69948abdb664038ba01c95e87fcb4eb/users/d4b43908501d4bc0951deb3463e67642/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:21:58.836869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.836869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.847426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7928dd93bf3044848d6469e4431f4cbd', 'request_id': 'req-017441f7-d476-4119-8026-13aa83e8fa75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.853345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7928dd93bf3044848d6469e4431f4cbd', 'request_id': 'req-017441f7-d476-4119-8026-13aa83e8fa75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.861399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7928dd93bf3044848d6469e4431f4cbd', 'request_id': 'req-017441f7-d476-4119-8026-13aa83e8fa75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.898475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c854f42-b032-40ca-888d-a14a725bd0e0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.956815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.957446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.957766 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.958036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] PATH_INFO: `/v3/users/c1ed0c9d7fe3460bbc1d00815f9e95bd` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.959508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ed0c9d7fe3460bbc1d00815f9e95bd)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.961495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.965539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] The token cache is being invalidate because project 7928dd93bf3044848d6469e4431f4cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:58.975110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.975110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-017441f7-d476-4119-8026-13aa83e8fa75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.975110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1059/2109] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:58 2022] DELETE /identity/v3/projects/7928dd93bf3044848d6469e4431f4cbd => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:58.975110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038dcf-81e2-48d6-b3d5-92100ea1b7d8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.979829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038dcf-81e2-48d6-b3d5-92100ea1b7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.980370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038dcf-81e2-48d6-b3d5-92100ea1b7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.018334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2f038dcf-81e2-48d6-b3d5-92100ea1b7d8 None None] MFA Rules not processed for user `d4b43908501d4bc0951deb3463e67642`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.019563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2f038dcf-81e2-48d6-b3d5-92100ea1b7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.085059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1060/2110] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.086405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.086746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.087084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.112808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.114025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.122894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1061/2111] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.123879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.125140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.184807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7fc53-8376-4654-9832-62cde05e4a7c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.188492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.189792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.190074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.190351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.198877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.200361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac831ce-1265-47a8-979f-5fd77aadc95b service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.209113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1062/2112] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:59.210352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.213571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.213571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.230473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.231672 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.241844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1063/2113] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.246539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.254148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ed0c9d7fe3460bbc1d00815f9e95bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.260031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ecb86-d4e6-4997-aba3-329dd0166c98 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.267650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.271347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.271347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.271347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.271347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:59.271347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.271715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.287889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1064/2114] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:21:59.290786 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.302149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01dbe-5190-481c-a352-84a9e416dc12 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.307437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.308071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.308352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.308622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.309304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.311437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.323783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ed0c9d7fe3460bbc1d00815f9e95bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.324603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ed0c9d7fe3460bbc1d00815f9e95bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.333226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ed0c9d7fe3460bbc1d00815f9e95bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.348101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c92d3e1a-8eeb-453e-b1a4-17dec1503aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ed0c9d7fe3460bbc1d00815f9e95bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.354550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1051/2115] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:58 2022] DELETE /identity/v3/users/c1ed0c9d7fe3460bbc1d00815f9e95bd => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:59.359915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.360269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.361327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.383991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.385524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.437518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1065/2116] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:59.441578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.443421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.488388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1052/2117] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.499123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.500922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.536737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50fa4-6faa-43cb-8dd3-d3d275c6de88 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.552452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.554367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.554835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.555308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.557085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.561444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.576661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca38e45-c337-4970-b4a5-8575711a45a3 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.597367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:59.652267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.652889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.653167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.653440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.664069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.665758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.674257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1066/2118] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:21:59.675253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.680032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1053/2119] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:59.697730 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca8e9-1dc0-4a88-8f05-8ede0f099564 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.711834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.712634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.712936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.713165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.713813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.716132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.731808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1067/2120] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:59.732824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.746462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa05e0-c7ab-4100-a7dc-ac4973525482 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.755499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.756419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.757695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.758195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.759762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.761934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037347e-0b2a-43df-af07-cccf12a8ab51 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.775280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1054/2121] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:59.779345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.803189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dc394-3575-4ede-b4e6-9b1c5ce41fe8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.817740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.820627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.821102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.821683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] PATH_INFO: `/v3/projects/c8ca8d208d474d27b9e0b0bda068528e/users/20ac57d592304654832256db715efdef/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.842333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ca8d208d474d27b9e0b0bda068528e, user_id=20ac57d592304654832256db715efdef, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.845179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.932600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1068/2122] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:59 2022] PUT /identity/v3/projects/c8ca8d208d474d27b9e0b0bda068528e/users/20ac57d592304654832256db715efdef/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:59.950565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.951050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.951485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.014841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] MFA Rules not processed for user `20ac57d592304654832256db715efdef`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:00.016250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:00.100114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1055/2123] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:00.297468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.301010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:00.313285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.314921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:00.376579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1b52-44ca-49a3-bd80-2d92bea2c8b1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.377343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5666f1-3b01-4cb8-9d2e-9a6553cd0b00 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.380940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.381468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.381621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.381811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.382080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.382222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.382480 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.382759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] PATH_INFO: `/v3/projects/d5674263f170421281aa132825e852a9` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.383987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5674263f170421281aa132825e852a9)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.386083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.391079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.396844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.415123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1056/2124] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:00.557621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5674263f170421281aa132825e852a9', 'request_id': 'req-211cd37f-4da8-45a6-9a34-88acf0661578'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:00.564932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5674263f170421281aa132825e852a9', 'request_id': 'req-211cd37f-4da8-45a6-9a34-88acf0661578'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:00.574508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5674263f170421281aa132825e852a9', 'request_id': 'req-211cd37f-4da8-45a6-9a34-88acf0661578'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:00.679180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] The token cache is being invalidate because project d5674263f170421281aa132825e852a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:00.679914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:00.680105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:00.681942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1069/2125] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:00 2022] DELETE /identity/v3/projects/d5674263f170421281aa132825e852a9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:00.685840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.696900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bb855-61b2-4f49-b62b-031e5e1e5a7e service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.753486 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.754108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.754380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.754647 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] PATH_INFO: `/v3/users/febdf476ca954e3dbbf228c1e28b5f55` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.755768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=febdf476ca954e3dbbf228c1e28b5f55)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.757840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.832754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:00.970348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febdf476ca954e3dbbf228c1e28b5f55'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.125426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febdf476ca954e3dbbf228c1e28b5f55'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.128016 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febdf476ca954e3dbbf228c1e28b5f55'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.143003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febdf476ca954e3dbbf228c1e28b5f55'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.164525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febdf476ca954e3dbbf228c1e28b5f55'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.171956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1057/2126] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:00 2022] DELETE /identity/v3/users/febdf476ca954e3dbbf228c1e28b5f55 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:01.314429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.319946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.341567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cd37f-4da8-45a6-9a34-88acf0661578 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.414358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.414999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.415273 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.415545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] PATH_INFO: `/v3/users/b1219819ed19468590c399e06ca8b1c4` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.416972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1219819ed19468590c399e06ca8b1c4)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.418882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.515845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.517365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.530872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2b64-1b74-4774-a3de-b31ac845e13f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.535755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.536531 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.536728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.536987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] PATH_INFO: `/v3/users/fea6987e86e4486d9f87ca6ea13a2621` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.538243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea6987e86e4486d9f87ca6ea13a2621)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.540257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.687280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1219819ed19468590c399e06ca8b1c4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.688064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1219819ed19468590c399e06ca8b1c4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.702962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1219819ed19468590c399e06ca8b1c4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.712961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1219819ed19468590c399e06ca8b1c4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.783547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea6987e86e4486d9f87ca6ea13a2621'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.793975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1219819ed19468590c399e06ca8b1c4'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.804851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1070/2127] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:01 2022] DELETE /identity/v3/users/b1219819ed19468590c399e06ca8b1c4 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:01.805900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.807648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.886410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbed12-697b-4218-9f81-a6ad3d96f322 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.891234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.892099 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.892525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.892841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] PATH_INFO: `/v3/projects/0b7063789ac94ea999602de1205e56fd` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.895008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7063789ac94ea999602de1205e56fd)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.897003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.913943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea6987e86e4486d9f87ca6ea13a2621'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.914821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea6987e86e4486d9f87ca6ea13a2621'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.930492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea6987e86e4486d9f87ca6ea13a2621'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.946291 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2931c266-a5ca-488f-a012-de05ff051c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea6987e86e4486d9f87ca6ea13a2621'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.953517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1058/2128] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:01 2022] DELETE /identity/v3/users/fea6987e86e4486d9f87ca6ea13a2621 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:01.957853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.958140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.958425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.982671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:01.984419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.051102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1059/2129] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:02.061805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.063456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.125265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7063789ac94ea999602de1205e56fd', 'request_id': 'req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.132354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7063789ac94ea999602de1205e56fd', 'request_id': 'req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.137817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d4df-29cf-49e9-b090-a312f281d6ab None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.142749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7063789ac94ea999602de1205e56fd', 'request_id': 'req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.196620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.197372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.197715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.198048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.271066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.272552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.283633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1060/2130] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:22:02.284475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.286044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.302319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fad2-16f0-4617-86f9-daa501f738c5 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.308877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.309721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.310050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.310689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] PATH_INFO: `/v3/projects/cf7d567a34404963a875cea01d9d8d1c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.311684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7d567a34404963a875cea01d9d8d1c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.314636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.331060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] The token cache is being invalidate because project 0b7063789ac94ea999602de1205e56fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:02.331634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.333416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3f8cc038-7813-4bc9-ac2f-15c79b056f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.335609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1071/2131] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:01 2022] DELETE /identity/v3/projects/0b7063789ac94ea999602de1205e56fd => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:02.342961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29869-d6c6-4d11-a3f0-52698b3c7da0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.343494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29869-d6c6-4d11-a3f0-52698b3c7da0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.344054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29869-d6c6-4d11-a3f0-52698b3c7da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.379107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-84d29869-d6c6-4d11-a3f0-52698b3c7da0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.380711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-84d29869-d6c6-4d11-a3f0-52698b3c7da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.404191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1072/2132] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:02.424105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ed27f-744d-4fd6-a541-6ae19406f69b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.424385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ed27f-744d-4fd6-a541-6ae19406f69b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.424662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ed27f-744d-4fd6-a541-6ae19406f69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.445661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-eb4ed27f-744d-4fd6-a541-6ae19406f69b None None] MFA Rules not processed for user `20ac57d592304654832256db715efdef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.446940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-eb4ed27f-744d-4fd6-a541-6ae19406f69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.492810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7d567a34404963a875cea01d9d8d1c', 'request_id': 'req-5477e885-c6dc-408a-9516-df60149eeb67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.499074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7d567a34404963a875cea01d9d8d1c', 'request_id': 'req-5477e885-c6dc-408a-9516-df60149eeb67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.510523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1073/2133] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:02.511139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.511424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.511701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.538765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.540110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.593798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:02.598419 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7d567a34404963a875cea01d9d8d1c', 'request_id': 'req-5477e885-c6dc-408a-9516-df60149eeb67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.611604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] The token cache is being invalidate because project cf7d567a34404963a875cea01d9d8d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:02.612067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.612843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.617885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1061/2134] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:02 2022] DELETE /identity/v3/projects/cf7d567a34404963a875cea01d9d8d1c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:02.619062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.622451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.652331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1074/2135] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:02.658152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.690133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555631-22f9-4d34-98c5-7da6f61b5bbc None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.698720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.701232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.701232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.701232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.701511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:02.701914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.704699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.722814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1075/2136] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:02.730447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.742839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af1f97-9dcf-4eb1-b600-e1ef6aa4154c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.747840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477e885-c6dc-408a-9516-df60149eeb67 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.749827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.750741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.751311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.751801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.752576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.754609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.754747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.755254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.755533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.755807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] PATH_INFO: `/v3/users/0af82ba96b5743be9cd44263fd0ca225` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.756988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af82ba96b5743be9cd44263fd0ca225)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.759234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.212230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1076/2137] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/projects => generated 375 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:03.214075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.217057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.321259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b71f-cb65-4327-9cdb-37593952464d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.328146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.329064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.329512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.334894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] PATH_INFO: `/v3/projects/0916502c9bec4047ae0a92b5fdfed7fe` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.334894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0916502c9bec4047ae0a92b5fdfed7fe)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.334894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.386956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af82ba96b5743be9cd44263fd0ca225'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.540844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af82ba96b5743be9cd44263fd0ca225'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.541724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af82ba96b5743be9cd44263fd0ca225'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.551212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af82ba96b5743be9cd44263fd0ca225'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.566534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af82ba96b5743be9cd44263fd0ca225'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.567712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0916502c9bec4047ae0a92b5fdfed7fe', 'request_id': 'req-4a8c796c-a240-4e57-9e45-ce1833eecd94'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.574437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1062/2138] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:02 2022] DELETE /identity/v3/users/0af82ba96b5743be9cd44263fd0ca225 => generated 0 bytes in 956 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:03.576111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0916502c9bec4047ae0a92b5fdfed7fe', 'request_id': 'req-4a8c796c-a240-4e57-9e45-ce1833eecd94'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.579188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.580845 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.587533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0916502c9bec4047ae0a92b5fdfed7fe', 'request_id': 'req-4a8c796c-a240-4e57-9e45-ce1833eecd94'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.667624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b708c-ffdb-4c0a-a7b6-8486cb087ed8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.672413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.673731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.674095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.674795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.675074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.679479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.705995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] The token cache is being invalidate because project 0916502c9bec4047ae0a92b5fdfed7fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:03.706874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.709040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:03.718418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1077/2139] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:03 2022] DELETE /identity/v3/projects/0916502c9bec4047ae0a92b5fdfed7fe => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:03.722603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.724637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.825189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c796c-a240-4e57-9e45-ce1833eecd94 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.834185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1063/2140] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:03 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:03.838745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.841627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.843818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.844963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.845411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.845831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] PATH_INFO: `/v3/users/3729db41a71641af9a47554a4beec611` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.847481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3729db41a71641af9a47554a4beec611)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.849707 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.858931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86dc5-7de0-4336-be5f-54bc45fa78e8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.866092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.866755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.867027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.867602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.868539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.870359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.883024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1064/2141] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:03.887230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.897884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db671fdd-7bd5-4486-b75c-6d4fa4f1ab25 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.903658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.904575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.904971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.905243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.906089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.908332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.922386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1065/2142] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:03.927246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.946957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198816-decc-4c02-b36f-af8c059c881d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.953167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.953844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.954125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.954400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] PATH_INFO: `/v3/projects/b4783ec021224963afc1ad9aad3e018e/users/516bcc7c8f014360bd3bbbeea5bde729/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.970496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4783ec021224963afc1ad9aad3e018e, user_id=516bcc7c8f014360bd3bbbeea5bde729, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.973306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:04.096406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1066/2143] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:03 2022] PUT /identity/v3/projects/b4783ec021224963afc1ad9aad3e018e/users/516bcc7c8f014360bd3bbbeea5bde729/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:04.097502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:04.099290 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.140132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3729db41a71641af9a47554a4beec611'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.140804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3729db41a71641af9a47554a4beec611'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.158385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3729db41a71641af9a47554a4beec611'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.168394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3729db41a71641af9a47554a4beec611'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.183951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff76d-e75f-42c6-a689-a192afbf6fbb admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:04.189088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:04.189726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.190058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.190340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] PATH_INFO: `/v3/projects/0975096b0cd04c4198e3117328d304b0` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.191461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0975096b0cd04c4198e3117328d304b0)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:04.193441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:04.291132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d3e17a82-b575-4533-981e-c7bf8126f75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3729db41a71641af9a47554a4beec611'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.299972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1078/2144] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:03 2022] DELETE /identity/v3/users/3729db41a71641af9a47554a4beec611 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:04.301605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.302066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.302394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.337462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:04.341578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] MFA Rules not processed for user `516bcc7c8f014360bd3bbbeea5bde729`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:04.344540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.359260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0975096b0cd04c4198e3117328d304b0', 'request_id': 'req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.366865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0975096b0cd04c4198e3117328d304b0', 'request_id': 'req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.451096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0975096b0cd04c4198e3117328d304b0', 'request_id': 'req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.481161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] The token cache is being invalidate because project 0975096b0cd04c4198e3117328d304b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:04.481503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.485374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad2ef132-1557-4fd7-97d8-5b37d77c42c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:04.489693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1067/2145] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:04 2022] DELETE /identity/v3/projects/0975096b0cd04c4198e3117328d304b0 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:04.495309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4237ea-5688-4e5b-9de7-7fc4dca65309 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.496002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4237ea-5688-4e5b-9de7-7fc4dca65309 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.497342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4237ea-5688-4e5b-9de7-7fc4dca65309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.520286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1079/2146] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:04.529559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:04.531824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.590689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7e4237ea-5688-4e5b-9de7-7fc4dca65309 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:04.594995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7e4237ea-5688-4e5b-9de7-7fc4dca65309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.693049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7cfe-35bd-408f-a41b-fd9ceb6afede None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:04.700405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:04.701044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.701323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.701592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.702737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:04.706418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:04.710598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1068/2147] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:04.725592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.725876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.726150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.757289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:04.759732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.773379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1069/2148] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:04.785214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:04.808518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9f34e-e278-4bfc-8c3f-5c4887b1e3ad None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:04.841904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:04.842837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:04.843299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:04.843796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:04.845447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:04.845977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:04.848148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:04.863514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1070/2149] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:04.868962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:04.878821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1080/2150] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/projects => generated 375 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:04.880435 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:04.882347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:04.885877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3f69d-46c6-4d94-8d8f-70d6e20944fe admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.036024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b813c-99db-410e-a073-e3ab5945cfb9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.043891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.045132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.045605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.046030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.046963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.051929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.052670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.052795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.053161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.053535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.054308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.056782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.200732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1071/2151] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/projects => generated 385 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:05.201812 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.206560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:05.208948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1081/2152] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:04 2022] POST /identity/v3/users => generated 288 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:05.212808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.234461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b9af-c4fe-4052-a315-8793926fab59 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.333569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.334185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.334461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.334749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f81cd-023d-493b-856a-3f068d987406 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.334976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.335613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.337824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.339550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.340212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.340490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.340765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] PATH_INFO: `/v3/projects/d906a1e33ec04eccb7979c20597b1c0a` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.341888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d906a1e33ec04eccb7979c20597b1c0a)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.343779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.509047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1082/2153] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:05 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:05.510003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.521631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff0c0-edd5-42c4-95ee-b3e857262c17 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.526567 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.527189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.527468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.530335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.530335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.530335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.545873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1083/2154] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:05.545873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.558153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978970-57ef-4dff-88d8-7c73919fb5a8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.574704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd906a1e33ec04eccb7979c20597b1c0a', 'request_id': 'req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:05.582256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd906a1e33ec04eccb7979c20597b1c0a', 'request_id': 'req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:05.643126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.643627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.643932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.644207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.644856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.646944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.662064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd906a1e33ec04eccb7979c20597b1c0a', 'request_id': 'req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:05.665772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1084/2155] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:05.669525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.690632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] The token cache is being invalidate because project d906a1e33ec04eccb7979c20597b1c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:05.690981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:05.693874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fe6af-6920-415b-b475-e8dce25423f6 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.694837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:05.697191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1072/2156] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:05 2022] DELETE /identity/v3/projects/d906a1e33ec04eccb7979c20597b1c0a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:05.698334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.699721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:05.716026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9717a0-26bc-48b0-a8a2-a88db6f74c4c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.721238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.723212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.725027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.725318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.726058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.727015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.727662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.728022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.728276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.728712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.728926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.730664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18c56-8188-4892-a94c-ad848dc9bc38 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.748178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1073/2157] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:05.748178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.748573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.748671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.753281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1085/2158] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:05.754637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.769843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed402b1-8110-4c40-a7b2-4cfafd26d4f2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.779601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:05.780917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:05.792382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1074/2159] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:05.793200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.794086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.796260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.796260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] PATH_INFO: `/v3/projects/20a6396fa5c840fc819d714bd83fe9ea/users/c8d4f6c6de864d27b72c3c30f6630067/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.796995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:05.798677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:05.814449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4b4a9-970b-4911-8b22-1253d021b025 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:05.820841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a6396fa5c840fc819d714bd83fe9ea, user_id=c8d4f6c6de864d27b72c3c30f6630067, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.823330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:05.823609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3f89-42ad-483c-ac00-5c526e2dcfa2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:05.824069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:05.824348 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:05.824622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] PATH_INFO: `/v3/projects/fb23743c19fb4c9daa51d90890ca8fd0/users/ba0cf3ef935d4aeb95f093cd7292cb98/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:05.841197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb23743c19fb4c9daa51d90890ca8fd0, user_id=ba0cf3ef935d4aeb95f093cd7292cb98, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:05.844345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9465b-2f4e-43d1-80b2-12ff0a7819c6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.002963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:06.009550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1075/2160] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:05 2022] PUT /identity/v3/projects/fb23743c19fb4c9daa51d90890ca8fd0/users/ba0cf3ef935d4aeb95f093cd7292cb98/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:06.019187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651a04-f740-4b60-ae75-98f827257025 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.019187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651a04-f740-4b60-ae75-98f827257025 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.019187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651a04-f740-4b60-ae75-98f827257025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.031982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1086/2161] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:05 2022] PUT /identity/v3/projects/20a6396fa5c840fc819d714bd83fe9ea/users/c8d4f6c6de864d27b72c3c30f6630067/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:06.034160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.034531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.034700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.068193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0e651a04-f740-4b60-ae75-98f827257025 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:06.069956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e651a04-f740-4b60-ae75-98f827257025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:06.077927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] MFA Rules not processed for user `ba0cf3ef935d4aeb95f093cd7292cb98`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:06.083926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:06.172998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1076/2162] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:06.175277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.175776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.176292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.182690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1087/2163] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:06.183828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.203923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fe0ea-60ab-4693-881d-4bf44b83b275 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.211326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.212393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.212669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.212944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.213607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:06.213905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.216149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.217283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] MFA Rules not processed for user `c8d4f6c6de864d27b72c3c30f6630067`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:06.218682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:06.233428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1088/2164] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:06.234325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.250652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f7e5-66ef-4670-b5a8-b9c6b28f5cb2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.260831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.261477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.261762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.262229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.262939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.265546 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.333035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1077/2165] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:06.333035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.344318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaec68-c992-47e1-91d5-0081ea399ade None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.350946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.351622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.352110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.352385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.353319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.355575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.400965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1089/2166] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:06.401703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.404975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:06.458331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1078/2167] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:06.459515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.508920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5fea-f610-4ec1-950a-9f3bcaad9c76 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.564550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afb04-f8f4-4390-92b1-6607e17cbb92 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.573146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.573960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.574240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.574505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.575480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.581908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.593370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.600941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.600941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.600941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.700498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1079/2168] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/users => generated 295 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:06.702664 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.703302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.708573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.720003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:06.722693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a653f0-1d51-49ed-9407-d512be254f5f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.727003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1090/2169] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:06.734428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.738880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.740318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.740597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.740877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.741875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.755716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.760497 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7912033-5c6b-4d3f-be2a-8ff6d2baddef service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.771809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.772584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.772867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.773905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.774325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.779221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.789205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1091/2170] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:06.806944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.829021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb85b17-af11-4c19-a704-8461a3b87bea admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.846023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.846449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.846728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.847056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] PATH_INFO: `/v3/projects/6d2165dd8d344c298facd412ebcd6f31/users/2738663a859548b880f3a9dffdd0319c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.884377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2165dd8d344c298facd412ebcd6f31, user_id=2738663a859548b880f3a9dffdd0319c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.888587 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.905800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1080/2171] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:06 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:06.910940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:06.934011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb189a-e1c7-4b77-9ee4-d8e8f0dbd858 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:06.944463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:06.946292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:06.946589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:06.946839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:06.948899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:06.953578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:06.998966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1092/2172] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:06 2022] PUT /identity/v3/projects/6d2165dd8d344c298facd412ebcd6f31/users/2738663a859548b880f3a9dffdd0319c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:07.004102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.008805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1081/2173] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:07.013797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.025010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c9-889d-477e-89d1-e0144d3d65f5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.083983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8721498-2ae2-4a8f-a000-f24c708d8e81 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.088833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.089044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.089460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.089610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.089783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.089888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.090014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.090114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.090639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.090739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.092345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.092458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.104039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1093/2174] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:07.107804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1082/2175] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:07.108109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.108361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.119202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1abf44-bba7-411a-9163-549bf8ae0db1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.119569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a38f8-5293-4b9d-8486-522f20157576 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.124223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.124613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.124904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.125182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.125354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.125625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] PATH_INFO: `/v3/projects/6d2165dd8d344c298facd412ebcd6f31/users/2738663a859548b880f3a9dffdd0319c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.125727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.126410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] PATH_INFO: `/v3/projects/30da6d7509a64c8b904d8220993af210/users/5d4092e9fbe24b2ca2f9c283cf22b68c/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.128454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2165dd8d344c298facd412ebcd6f31, user_id=2738663a859548b880f3a9dffdd0319c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.140711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61522-df51-467c-ba74-d259a2da66d5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.150395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30da6d7509a64c8b904d8220993af210, user_id=5d4092e9fbe24b2ca2f9c283cf22b68c, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.154050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1094/2176] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:07 2022] PUT /identity/v3/projects/6d2165dd8d344c298facd412ebcd6f31/users/2738663a859548b880f3a9dffdd0319c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:07.160487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758a99-6c9d-4067-9837-953e5d5f93e4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.166189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.166723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.167136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.210125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] MFA Rules not processed for user `2738663a859548b880f3a9dffdd0319c`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:07.211223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.217301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:07.257410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1083/2177] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:07 2022] PUT /identity/v3/projects/30da6d7509a64c8b904d8220993af210/users/5d4092e9fbe24b2ca2f9c283cf22b68c/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:07.262938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.263239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.263516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.293966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:07.295345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.356215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1095/2178] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:07.386394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.388698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.405171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1084/2179] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:07.420325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.422538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.484037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ecfb-00da-4f98-b361-a96e402bb004 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.521064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0385626-0b97-4a54-b7bf-dd87ebff0687 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.527213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.528192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.528495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.528800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.539331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.541095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.551725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1085/2180] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:07.567187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.568010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.568390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.568879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.571504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.685314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.685314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:07.701486 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1096/2181] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:07.783549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:07.788707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:07.822673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca87670-df0c-4024-8c03-b82007328a44 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:07.833939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:07.834711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:07.835044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:07.835509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] PATH_INFO: `/v3/users/f97d0910139b4610b792092b6133596d` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:07.837116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97d0910139b4610b792092b6133596d)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:07.840048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:08.151013 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97d0910139b4610b792092b6133596d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:08.312406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97d0910139b4610b792092b6133596d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:08.313012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97d0910139b4610b792092b6133596d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:08.325421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97d0910139b4610b792092b6133596d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:08.354231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-1204e085-eb21-4474-a28e-6087a082b861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97d0910139b4610b792092b6133596d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:08.363214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1086/2182] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:07 2022] DELETE /identity/v3/users/f97d0910139b4610b792092b6133596d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:08.766296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:08.779572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d436-a15d-4f3e-8a41-69aef23a20af service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:08.893702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:08.894917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:08.895286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:08.895774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] PATH_INFO: `/v3/projects/f991d9be87e5494588cafcdcd6841e7d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:08.897126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f991d9be87e5494588cafcdcd6841e7d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:08.899263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:08.983414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:09.125619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f991d9be87e5494588cafcdcd6841e7d', 'request_id': 'req-a6e1f37e-11df-47ad-bd18-523b025a441a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:09.133275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f991d9be87e5494588cafcdcd6841e7d', 'request_id': 'req-a6e1f37e-11df-47ad-bd18-523b025a441a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:09.139962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f991d9be87e5494588cafcdcd6841e7d', 'request_id': 'req-a6e1f37e-11df-47ad-bd18-523b025a441a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:09.189460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:09.306524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] The token cache is being invalidate because project f991d9be87e5494588cafcdcd6841e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:09.307023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:09.308122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a6e1f37e-11df-47ad-bd18-523b025a441a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:09.310921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1097/2183] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:08 2022] DELETE /identity/v3/projects/f991d9be87e5494588cafcdcd6841e7d => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:09.316622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.317092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.317470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.338905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:09.340281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:09.414371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1087/2184] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:09.429840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.430226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.430591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.455619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:09.457075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:09.465629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1098/2185] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:09.473454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.486805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8b71-f983-442e-8ac1-8af2ce6d64ee None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:09.491707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:09.492365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.492772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.493054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.494097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:09.495097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:09.497203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c008e83-60ec-48d5-b0f1-2d8f65ca334b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:09.514494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1088/2186] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:09.519074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.530560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3995a65-5f1a-48b4-99f4-7f5cae9ac0d7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:09.541976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:09.542692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.543044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.543381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.544673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:09.546212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:09.604762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.605148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.605424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.643819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] MFA Rules not processed for user `5d4092e9fbe24b2ca2f9c283cf22b68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:09.646961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:09.694191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1099/2187] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:09 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:09.698877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.750652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1089/2188] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:09.766715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.771897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:09.817187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26952-250c-47a2-aff6-57f1156f3e5f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:09.822377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:09.823092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.823430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.823759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.826795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:09.826795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:09.845434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5a118-5f2e-460c-8e43-9818ea4a06b0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:09.856945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:09.921773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1100/2189] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:09 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:09.924790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:09.925769 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.926284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.926585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.926743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.939699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b92695-1bac-487d-8977-63cdf290436e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:09.949896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:09.950527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:09.950805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:09.951275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:09.951752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:09.953456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:09.968369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1101/2190] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:09.970793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:09.992182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a322717-c514-4fd3-b171-5d9da29c6300 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:10.001811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:10.002458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.002721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.002986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.003622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.005323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.015877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.028855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.029016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1102/2191] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:10.029016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:10.029016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:10.029282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1090/2192] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:22:10.032050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:10.036443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c11b4-9fa2-4117-9fa3-7228638d4263 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:10.044304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:10.044938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.045362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.045616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.047370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:10.048570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071330b-92cf-42bc-9f71-8e96a7829777 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:10.053847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:10.054636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.054978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.055313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] PATH_INFO: `/v3/projects/8e73450992ac46049edabd7000d44bde/users/9d7e9a2846cc4ff3b64a8551a7a733a2/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.083425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e73450992ac46049edabd7000d44bde, user_id=9d7e9a2846cc4ff3b64a8551a7a733a2, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.086142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.137851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.139387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098861f-87dc-4044-8057-58b1d193818e service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.154874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1103/2193] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:10.166272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1091/2194] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:10 2022] PUT /identity/v3/projects/8e73450992ac46049edabd7000d44bde/users/9d7e9a2846cc4ff3b64a8551a7a733a2/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:10.174001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.176232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.176884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.201426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:10.202982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:10.273383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1104/2195] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:10.283708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:10.285579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:10.352058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de41d9-10df-4556-b17f-b6e9efe28c08 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:10.356402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:10.357025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.357302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.357571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.369376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.371903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b238a4-0419-45e6-b302-73ccb5fd8788 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.382848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1092/2196] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:10.639808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:11.319723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:12.170138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:12.417368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:13.469040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:13.539110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632a653-b22a-4f12-b0c5-544ddde32227 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:13.544049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:13.544731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:13.545009 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:13.545284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:13.547382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:13.644457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:13.645969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700aee6-3720-42d8-8994-c11bf8a48475 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:13.659320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1105/2197] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:14.300437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:14.301436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:14.302012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:14.336752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] MFA Rules not processed for user `5d4092e9fbe24b2ca2f9c283cf22b68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:14.338460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:14.451623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1093/2198] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:14.874545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:17.124576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:17.667317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:17.667603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:17.667948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:17.743088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] MFA Rules not processed for user `9d7e9a2846cc4ff3b64a8551a7a733a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:17.744789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:17.836215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1106/2199] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:17.852225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:17.861332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:17.942693 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be8be5-3629-48bd-8c27-773cc252d2d2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:17.948649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:17.949271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:17.949571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:17.949865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:17.958699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:17.960371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477a7b1-70bb-4362-bad6-429f61488d67 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:17.970362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1094/2200] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:18.032899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:18.769564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:18.771845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:18.788163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8cda4-2909-4ae8-b8b9-5f7f44610256 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:18.797319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:18.797319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:18.798221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:18.798606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] PATH_INFO: `/v3/users/c8d4f6c6de864d27b72c3c30f6630067` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:18.800073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d4f6c6de864d27b72c3c30f6630067)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:18.802212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:19.128446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d4f6c6de864d27b72c3c30f6630067'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:19.129367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d4f6c6de864d27b72c3c30f6630067'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:19.148772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d4f6c6de864d27b72c3c30f6630067'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:19.161288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d4f6c6de864d27b72c3c30f6630067'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:19.297088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d4f6c6de864d27b72c3c30f6630067'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:19.312088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1107/2201] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:18 2022] DELETE /identity/v3/users/c8d4f6c6de864d27b72c3c30f6630067 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:19.318025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:19.318902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:19.319318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:19.373118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:19.374440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:19.535984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1095/2202] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:19.551028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:19.553766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:19.652675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4289fa-835d-437a-86d1-987bdf8acae7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:19.656669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:19.657845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:19.658116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:19.658388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:19.668357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:19.669741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:19.681105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1108/2203] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:19.902628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:20.197915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:20.199649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:20.212737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20b29d-b8f7-42c0-993c-3f65f6fe595b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:20.220203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:20.221289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:20.221713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:20.222146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] PATH_INFO: `/v3/users/516bcc7c8f014360bd3bbbeea5bde729` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:20.223568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516bcc7c8f014360bd3bbbeea5bde729)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:20.225901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:20.506800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:20.508438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:20.544928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d9686-474f-4b64-9db0-5373d29d4f12 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:20.571413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:20.576514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:20.576799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:20.578488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516bcc7c8f014360bd3bbbeea5bde729'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.581145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] PATH_INFO: `/v3/projects/20a6396fa5c840fc819d714bd83fe9ea` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:20.586608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a6396fa5c840fc819d714bd83fe9ea)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:20.590862 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:20.684446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516bcc7c8f014360bd3bbbeea5bde729'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.685095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516bcc7c8f014360bd3bbbeea5bde729'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.700930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516bcc7c8f014360bd3bbbeea5bde729'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.730130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b5135f75-7e17-40e1-8557-9a30d5c77974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516bcc7c8f014360bd3bbbeea5bde729'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.739651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1096/2204] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:20 2022] DELETE /identity/v3/users/516bcc7c8f014360bd3bbbeea5bde729 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:20.743712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef440-d612-40d7-a3a5-12feed8a4e75 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:20.744064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef440-d612-40d7-a3a5-12feed8a4e75 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:20.744890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef440-d612-40d7-a3a5-12feed8a4e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:20.790187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-302ef440-d612-40d7-a3a5-12feed8a4e75 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:20.797181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-302ef440-d612-40d7-a3a5-12feed8a4e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:20.802588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a6396fa5c840fc819d714bd83fe9ea', 'request_id': 'req-d91ff8e3-faf2-49b5-8c96-5b687effbb14'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.820890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a6396fa5c840fc819d714bd83fe9ea', 'request_id': 'req-d91ff8e3-faf2-49b5-8c96-5b687effbb14'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.833353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a6396fa5c840fc819d714bd83fe9ea', 'request_id': 'req-d91ff8e3-faf2-49b5-8c96-5b687effbb14'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.941620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] The token cache is being invalidate because project 20a6396fa5c840fc819d714bd83fe9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:20.941620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.942314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:20.944750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1097/2205] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:20.949052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1109/2206] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:20 2022] DELETE /identity/v3/projects/20a6396fa5c840fc819d714bd83fe9ea => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:20.967109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c534-b6dc-4528-babc-226070126d8f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:20.967916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c534-b6dc-4528-babc-226070126d8f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:20.968438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c534-b6dc-4528-babc-226070126d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:20.980860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:20.982673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.030227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4384c534-b6dc-4528-babc-226070126d8f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:21.036897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4384c534-b6dc-4528-babc-226070126d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.054674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1098/2207] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:21.063475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.063759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.064131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.087573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:21.088876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.097853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1099/2208] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:21.102341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.109350 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff8e3-faf2-49b5-8c96-5b687effbb14 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.130914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a4ce-b990-42fd-8de3-d9dbc11f696c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.131103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.131103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.131103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.131103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.133826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.148101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.149113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.149578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.150043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.150786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:21.151210 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.170139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.170485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.170485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.181470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1100/2209] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:21.182525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.185650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1110/2210] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:22:21.199457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59499-9454-4d05-b2c3-9cf241513fce admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.204757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264bf67-54f5-4f89-b32a-c9c376e9e326 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.304810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1101/2211] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:21 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:21.312649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.324925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff03c6-c562-4d89-a47d-0cadb181ba54 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.403694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.404489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.404866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.405224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.405926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.407998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.460530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.461148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.461596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.486833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] MFA Rules not processed for user `9d7e9a2846cc4ff3b64a8551a7a733a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:21.488173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.523159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1111/2212] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:21 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:21.527883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.542201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce0890-7ec1-494f-bef4-888cd88f621e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.549972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.551228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.551895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.552347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.553845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.555506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.572766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1112/2213] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:21.578205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.597366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00a56-ef9e-457e-bde2-d75128996181 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.608019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.608714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.610765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.612442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.613199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.621435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1102/2214] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:21.621865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021819d-5e8c-4db1-8f69-d06a5dc0a306 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.630648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1113/2215] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:21.634644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:21.649355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc006-8f81-4e97-ab63-9201b1ab39a9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:21.654926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:21.658692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.658692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.658692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] PATH_INFO: `/v3/projects/ce37ac421e51434da853e6f5d8c2af4a/users/4e93acdde18f4b30bd017512a113e16d/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.679024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce37ac421e51434da853e6f5d8c2af4a, user_id=4e93acdde18f4b30bd017512a113e16d, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:21.683380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:21.833924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:21.834262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:21.834544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:21.837319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1103/2216] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:21 2022] PUT /identity/v3/projects/ce37ac421e51434da853e6f5d8c2af4a/users/4e93acdde18f4b30bd017512a113e16d/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:21.877946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] MFA Rules not processed for user `4e93acdde18f4b30bd017512a113e16d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:21.879386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:21.999510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1114/2217] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:22.013708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:22.015416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:22.085223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81968f-7070-46b4-b82e-a58853b3dd73 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:22.086203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:22.086927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:22.087209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:22.087483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:22.102472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:22.104704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:22.124526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:22.130814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1104/2218] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:22.133861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:22.136791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:22.239280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17d15c-649d-4a78-b42a-9fb4525c011d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:22.245441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:22.246364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:22.246737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:22.247195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] PATH_INFO: `/v3/projects/b4783ec021224963afc1ad9aad3e018e` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:22.250004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4783ec021224963afc1ad9aad3e018e)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:22.252990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:22.474332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4783ec021224963afc1ad9aad3e018e', 'request_id': 'req-cdf89c15-4cf7-4666-9e25-2f440bd69a40'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:22.480157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4783ec021224963afc1ad9aad3e018e', 'request_id': 'req-cdf89c15-4cf7-4666-9e25-2f440bd69a40'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:22.490645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4783ec021224963afc1ad9aad3e018e', 'request_id': 'req-cdf89c15-4cf7-4666-9e25-2f440bd69a40'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:22.617808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] The token cache is being invalidate because project b4783ec021224963afc1ad9aad3e018e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:22.621109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:22.621109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:22.621109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1115/2219] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:22 2022] DELETE /identity/v3/projects/b4783ec021224963afc1ad9aad3e018e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:22.626404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:22.673598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183037b6-fb8e-41d1-8425-4ede90f38504 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:22.759240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:22.760874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:22.761156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:22.761434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] PATH_INFO: `/v3/users/ba0cf3ef935d4aeb95f093cd7292cb98` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:22.762688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0cf3ef935d4aeb95f093cd7292cb98)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:22.764745 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:23.036297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0cf3ef935d4aeb95f093cd7292cb98'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:23.120325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0cf3ef935d4aeb95f093cd7292cb98'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:23.121480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0cf3ef935d4aeb95f093cd7292cb98'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:23.135842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0cf3ef935d4aeb95f093cd7292cb98'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:23.155080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0cf3ef935d4aeb95f093cd7292cb98'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:23.165804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1105/2220] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:22 2022] DELETE /identity/v3/users/ba0cf3ef935d4aeb95f093cd7292cb98 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:23.802894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:23.804396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:23.819243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89c15-4cf7-4666-9e25-2f440bd69a40 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:23.880958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:23.881596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:23.881874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:23.882147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] PATH_INFO: `/v3/projects/fb23743c19fb4c9daa51d90890ca8fd0` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:23.883413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb23743c19fb4c9daa51d90890ca8fd0)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:23.885366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:23.981175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:24.054831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb23743c19fb4c9daa51d90890ca8fd0', 'request_id': 'req-a83f2138-44ee-4281-87a8-bbd512ce9fea'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.060311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb23743c19fb4c9daa51d90890ca8fd0', 'request_id': 'req-a83f2138-44ee-4281-87a8-bbd512ce9fea'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.067346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb23743c19fb4c9daa51d90890ca8fd0', 'request_id': 'req-a83f2138-44ee-4281-87a8-bbd512ce9fea'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.240680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] The token cache is being invalidate because project fb23743c19fb4c9daa51d90890ca8fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:24.240981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.241666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.243227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1116/2221] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:23 2022] DELETE /identity/v3/projects/fb23743c19fb4c9daa51d90890ca8fd0 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:24.247974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:24.249478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:24.263226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f51eb-1e46-470f-8ec7-01a089f01876 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:24.323446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:24.324130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:24.324408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:24.324679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] PATH_INFO: `/v3/users/2738663a859548b880f3a9dffdd0319c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:24.325918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2738663a859548b880f3a9dffdd0319c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:24.327999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:24.551381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2738663a859548b880f3a9dffdd0319c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.651026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2738663a859548b880f3a9dffdd0319c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.651707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2738663a859548b880f3a9dffdd0319c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.666665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2738663a859548b880f3a9dffdd0319c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.681347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-908e436a-4e26-4e71-92f8-0f25faf9b2c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2738663a859548b880f3a9dffdd0319c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:24.687891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1106/2222] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:24 2022] DELETE /identity/v3/users/2738663a859548b880f3a9dffdd0319c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:25.224502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:25.226168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:25.247149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2138-44ee-4281-87a8-bbd512ce9fea admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:25.320227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:25.320787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.321070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.321343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] PATH_INFO: `/v3/projects/6d2165dd8d344c298facd412ebcd6f31` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.322522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2165dd8d344c298facd412ebcd6f31)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:25.324689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:25.454186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2165dd8d344c298facd412ebcd6f31', 'request_id': 'req-dfed1b92-752a-414f-9065-38cc2a4dc52d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:25.460121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2165dd8d344c298facd412ebcd6f31', 'request_id': 'req-dfed1b92-752a-414f-9065-38cc2a4dc52d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:25.466972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2165dd8d344c298facd412ebcd6f31', 'request_id': 'req-dfed1b92-752a-414f-9065-38cc2a4dc52d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:25.552934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] The token cache is being invalidate because project 6d2165dd8d344c298facd412ebcd6f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:25.553576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:25.554295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dfed1b92-752a-414f-9065-38cc2a4dc52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:25.555882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1117/2223] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:25 2022] DELETE /identity/v3/projects/6d2165dd8d344c298facd412ebcd6f31 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:25.561030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.561525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.561652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.582143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:25.583508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:25.647597 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1107/2224] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:25.657179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.657580 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.657857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.679309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:25.680573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:25.688785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1118/2225] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:25.693142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:25.703063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c8d7-e66c-48ce-bf16-e93bb3dd69c0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:25.707804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:25.708448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.708721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.708991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.709669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:25.709971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:25.711604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:25.725502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1108/2226] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:25.729199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:25.739654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75500d35-deb4-48f7-b4d5-3d1892d24a05 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:25.746564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:25.747191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.747473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.747749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.748448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:25.750444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:25.765748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:25.830164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1119/2227] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:25 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:25.832600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:25.846858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614288-2b9a-46ca-bff4-7fee37dac82f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:25.915716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:25.916592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:25.916869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:25.917794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:25.918461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:25.920437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.078342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1109/2228] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:25 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:26.082853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.093743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05885b-b534-43f5-a084-2aa21106ff8a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.098682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.099292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.099571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.099840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.100511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.103312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.117464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1120/2229] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:26.122070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.134007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22523dca-48fb-4c8b-acac-a0de482cf06a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.139048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.139825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.140405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.140820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.142119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.144001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95d47a-7662-4435-9eb3-ced9b92911a6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.158005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1110/2230] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:26.165070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.173909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f8e98-8c01-4b63-bd3a-e8f84e7d3319 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.178658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.179391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.179680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.179978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] PATH_INFO: `/v3/projects/cf3b20447eb44ade98f9ba3cdff335a6/users/6ad614c818764c8694f55affeec29408/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.197929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf3b20447eb44ade98f9ba3cdff335a6, user_id=6ad614c818764c8694f55affeec29408, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.200714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.313873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1121/2231] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:26 2022] PUT /identity/v3/projects/cf3b20447eb44ade98f9ba3cdff335a6/users/6ad614c818764c8694f55affeec29408/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:26.324821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.325101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.325372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.358356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] MFA Rules not processed for user `6ad614c818764c8694f55affeec29408`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:26.360215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:26.432260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1111/2232] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:26.437354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.504498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62dd12-543b-4098-9371-6eeb8aae09c8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.509487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.510115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.510393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.510668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.511330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.513499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.611460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1122/2233] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:26 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:26.615630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.628000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ffe80-31f3-4692-9898-61f9aa5ed014 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.691472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.692123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.692397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.692665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.693309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.695237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.800563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1112/2234] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:26 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:26.804798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.817145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71f4d5-69b0-4c37-abf9-5d055be5867e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.822573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.823272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.823552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.823818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.824520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.826128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.837996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1123/2235] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:26.842138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.852424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-6bc6-4a76-a6a8-fd57e0e3daa8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.857993 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.858743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.858956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.859191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.860529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.862557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f8dc2-3312-4288-962a-84e47d39cd67 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:26.876802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1113/2236] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:26.877974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:26.888657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b96-1877-45b9-a63b-4b9310d49052 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:26.893366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:26.894068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:26.894366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:26.894634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] PATH_INFO: `/v3/projects/5b899894e35c4187b290e45baa255b68/users/4344c4d1f2ca4739b05a877e7aa5e6a2/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:26.911102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b899894e35c4187b290e45baa255b68, user_id=4344c4d1f2ca4739b05a877e7aa5e6a2, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:26.913826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.028085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1124/2237] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:26 2022] PUT /identity/v3/projects/5b899894e35c4187b290e45baa255b68/users/4344c4d1f2ca4739b05a877e7aa5e6a2/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:27.040351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.040641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.040921 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.054966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:27.071837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] MFA Rules not processed for user `4344c4d1f2ca4739b05a877e7aa5e6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:27.073123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:27.138389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1114/2238] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:27.143091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.209402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767e0f9-5e7c-47fd-a1b5-85ac40ca3016 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.214417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.215037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.215359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.215631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.216307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.218251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.313925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1125/2239] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:27 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:27.318933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.331250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2c1b-0e2f-40f2-8da5-2f82919a4531 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.394711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:27.411882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.411882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.411882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.411882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.412480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.412480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.514318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1115/2240] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:27 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:27.518831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.530312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b848-b14b-4807-9629-a0189e73f038 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.535796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.536508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.536831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.537105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.537754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.539356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.551207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1126/2241] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:27.555569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.565827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8099-7491-455f-9f16-fadd90347b2b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.570415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.571062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.571335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.571604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] PATH_INFO: `/v3/projects/51b0543ac9f840b7a390a838f3557a37/users/046da16557e444cb9b4e518bab431b43/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.587047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b0543ac9f840b7a390a838f3557a37, user_id=046da16557e444cb9b4e518bab431b43, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.589776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.680220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1116/2242] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:27 2022] PUT /identity/v3/projects/51b0543ac9f840b7a390a838f3557a37/users/046da16557e444cb9b4e518bab431b43/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:27.685175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.697721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1960c2-14f8-44c4-9df0-33c326894093 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.776130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.776744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.777019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.777297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.777939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.779537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14964-a16e-4755-977f-5bf1a90fb94c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.792609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1127/2243] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:27.805369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.817827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e7c8-1c72-4f4f-932b-50df786120c4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:27.822692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:27.823345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.823622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.823916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] PATH_INFO: `/v3/projects/51b0543ac9f840b7a390a838f3557a37/users/046da16557e444cb9b4e518bab431b43/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.825863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b0543ac9f840b7a390a838f3557a37, user_id=046da16557e444cb9b4e518bab431b43, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:27.829087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:27.840778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1117/2244] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:27 2022] PUT /identity/v3/projects/51b0543ac9f840b7a390a838f3557a37/users/046da16557e444cb9b4e518bab431b43/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:27.855405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:27.855689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:27.856001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:27.885275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] MFA Rules not processed for user `046da16557e444cb9b4e518bab431b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:27.886410 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:27.961082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1128/2245] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:27.979662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:27.979662 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:28.054738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7022-a588-43af-9566-c1aa0abe34b1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:28.060143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:28.060781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:28.061058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:28.062114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:28.156857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:28.158340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:28.171441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1118/2246] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:22:29.463511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:30.167285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:30.260384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:30.270689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71dd99-ae8f-461e-be3b-51cffbdfbef2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:30.274452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:30.275121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:30.275396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:30.275671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:30.337154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:30.344770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af57ce7-325c-4824-bbf9-6985cf7b6cc1 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:30.349367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1129/2247] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:31.542280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:31.748041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:32.651055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:33.169825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:33.405034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:33.670674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:33.680952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e26faa-fd3a-4da9-8a0c-15204bb294db service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:33.686213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:33.686959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:33.687230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:33.688120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:33.699277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:33.700912 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:33.737001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1119/2248] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:35.908467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:35.964931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:36.536728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:36.588066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:37.227633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:37.284898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:38.184150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:38.743197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c1c7-8c82-4218-a035-516782d34ee9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:38.743197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c1c7-8c82-4218-a035-516782d34ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:38.743844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c1c7-8c82-4218-a035-516782d34ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:38.763084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8029c1c7-8c82-4218-a035-516782d34ee9 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:38.764394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8029c1c7-8c82-4218-a035-516782d34ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:38.821993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1130/2249] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:38.866618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:38.868468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:38.943345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a0ec5-8a7f-46ca-b5bc-36baa41ef5cc service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:38.950426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:38.951055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:38.951331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:38.951603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:38.960695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:38.962613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:38.976129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:38.976129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1120/2250] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:22:39.988565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06aa6-a94b-4c4e-9ed7-f507f5239e48 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.988565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06aa6-a94b-4c4e-9ed7-f507f5239e48 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.988565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06aa6-a94b-4c4e-9ed7-f507f5239e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.000902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-40a06aa6-a94b-4c4e-9ed7-f507f5239e48 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:40.002127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-40a06aa6-a94b-4c4e-9ed7-f507f5239e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:40.009443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1131/2251] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:22:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:40.023055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.035350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79a2d-55f3-4e2d-9c98-7d3b51be2d9d service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.040813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.041333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.041633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.041884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.050794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.056098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.060837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1121/2252] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:22:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:22:40.188654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:40.570698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:41.041653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f3efa-c04e-4d6f-baf0-4bd50d1ad769 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.042173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f3efa-c04e-4d6f-baf0-4bd50d1ad769 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.042327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f3efa-c04e-4d6f-baf0-4bd50d1ad769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.062914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b44f3efa-c04e-4d6f-baf0-4bd50d1ad769 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:41.064295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b44f3efa-c04e-4d6f-baf0-4bd50d1ad769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:41.072026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1132/2253] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:22:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:41.085116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.096519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ad38-a542-4782-a7a3-9e9be14d58e4 service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.102032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.102649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.102928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.103201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.112183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.113808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.122538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1122/2254] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:22:41.432096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:41.662741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:41.888428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.888716 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.888993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.912296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:41.913565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:41.929668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1133/2255] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:22:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:41.954558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:42.000489 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156e32-e81d-445f-8622-d190438cf291 service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:42.010255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:42.011089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.011377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.011653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.022078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:42.024435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79559-4e20-4b81-880b-d8a6ba5320a3 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:42.034641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1123/2256] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:22:43.931070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:44.279602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:45.943302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.945329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.027151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00410f-da8d-4dad-8c65-5ea920303436 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.032691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.033314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.033588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.033865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] PATH_INFO: `/v3/users/20ac57d592304654832256db715efdef` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.035144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ac57d592304654832256db715efdef)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.037254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.260199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ac57d592304654832256db715efdef'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.261082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ac57d592304654832256db715efdef'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.276272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ac57d592304654832256db715efdef'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.285592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ac57d592304654832256db715efdef'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.369534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ac57d592304654832256db715efdef'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.376822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1134/2257] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:45 2022] DELETE /identity/v3/users/20ac57d592304654832256db715efdef => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:46.381605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.382111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.382214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.402282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:46.403473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.462527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1124/2258] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:46.472426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.473842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.531896 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16f86-1b3d-4b29-8234-6ccd3d37ce04 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.535495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.536162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.536442 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.536713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.547081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.548598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.558080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1135/2259] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:46.593740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:46.674057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:46.795578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.797678 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.811799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3765339-bc2b-410e-9206-16819436bf6e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.847069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.848999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.849375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.849913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] PATH_INFO: `/v3/projects/c8ca8d208d474d27b9e0b0bda068528e` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.852256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ca8d208d474d27b9e0b0bda068528e)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.854417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.992005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ca8d208d474d27b9e0b0bda068528e', 'request_id': 'req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:46.998649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ca8d208d474d27b9e0b0bda068528e', 'request_id': 'req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:47.072935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ca8d208d474d27b9e0b0bda068528e', 'request_id': 'req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:47.086802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] The token cache is being invalidate because project c8ca8d208d474d27b9e0b0bda068528e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:47.087135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:47.087829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3b7564cf-4362-4fe4-bc58-a4b3f1ea8b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:47.089627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1125/2260] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:46 2022] DELETE /identity/v3/projects/c8ca8d208d474d27b9e0b0bda068528e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:47.597934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:48.209877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:48.211487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:48.225663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950aaa-6b4f-40f1-afdd-50c256226cb9 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:48.254714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:48.289550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:48.290195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:48.290491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:48.290823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] PATH_INFO: `/v3/users/6ad614c818764c8694f55affeec29408` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:48.292010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad614c818764c8694f55affeec29408)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:48.293952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:48.498850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad614c818764c8694f55affeec29408'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:48.500094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad614c818764c8694f55affeec29408'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:48.512779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad614c818764c8694f55affeec29408'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:48.520347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad614c818764c8694f55affeec29408'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:48.576060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad614c818764c8694f55affeec29408'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:48.583000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1136/2261] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:48 2022] DELETE /identity/v3/users/6ad614c818764c8694f55affeec29408 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:48.587521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:48.587796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:48.588078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:48.607990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:48.609721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:48.669669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1126/2262] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:48.679700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:48.681141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:48.740803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd96f-f4dc-4dae-829b-f41408375fdd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:48.744693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:48.745299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:48.745579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:48.745846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:48.755679 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:48.757137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:48.766551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1137/2263] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:48.788133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:48.931180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:49.199023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:49.281003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:49.283958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:49.324190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88b63-4aa9-4292-b6be-25177b68d465 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:49.331223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:49.332074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:49.332603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:49.333099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] PATH_INFO: `/v3/projects/cf3b20447eb44ade98f9ba3cdff335a6` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:49.334576 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf3b20447eb44ade98f9ba3cdff335a6)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:49.337204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:49.496053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf3b20447eb44ade98f9ba3cdff335a6', 'request_id': 'req-cf7f2f74-615e-4485-88ff-4b65403980ee'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.502174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf3b20447eb44ade98f9ba3cdff335a6', 'request_id': 'req-cf7f2f74-615e-4485-88ff-4b65403980ee'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.591117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf3b20447eb44ade98f9ba3cdff335a6', 'request_id': 'req-cf7f2f74-615e-4485-88ff-4b65403980ee'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.604719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] The token cache is being invalidate because project cf3b20447eb44ade98f9ba3cdff335a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:49.605011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.605673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.607541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1127/2264] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:49 2022] DELETE /identity/v3/projects/cf3b20447eb44ade98f9ba3cdff335a6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:49.612168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:49.621842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102ef8d-446d-454c-9425-07e825205939 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:49.683710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:49.684514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:49.685149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:49.685467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] PATH_INFO: `/v3/users/4344c4d1f2ca4739b05a877e7aa5e6a2` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:49.686779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4344c4d1f2ca4739b05a877e7aa5e6a2)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:49.688726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:49.810726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:49.864947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4344c4d1f2ca4739b05a877e7aa5e6a2'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.865693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4344c4d1f2ca4739b05a877e7aa5e6a2'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.878412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4344c4d1f2ca4739b05a877e7aa5e6a2'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.885729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4344c4d1f2ca4739b05a877e7aa5e6a2'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.901708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:49.964269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4344c4d1f2ca4739b05a877e7aa5e6a2'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:49.971198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1138/2265] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:49 2022] DELETE /identity/v3/users/4344c4d1f2ca4739b05a877e7aa5e6a2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:50.062702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:50.462972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:50.466089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:50.466483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:50.481686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2f74-615e-4485-88ff-4b65403980ee admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:50.513189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:50.557147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:50.557746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:50.557994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:50.558264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] PATH_INFO: `/v3/projects/5b899894e35c4187b290e45baa255b68` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:50.559460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b899894e35c4187b290e45baa255b68)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:50.561472 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:50.668383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b899894e35c4187b290e45baa255b68', 'request_id': 'req-a15da203-3056-4ac7-aa0a-3859a7a9aca5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:50.674035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b899894e35c4187b290e45baa255b68', 'request_id': 'req-a15da203-3056-4ac7-aa0a-3859a7a9aca5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:50.740080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b899894e35c4187b290e45baa255b68', 'request_id': 'req-a15da203-3056-4ac7-aa0a-3859a7a9aca5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:50.754267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] The token cache is being invalidate because project 5b899894e35c4187b290e45baa255b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:50.754532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:50.755241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:50.756916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1128/2266] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:50 2022] DELETE /identity/v3/projects/5b899894e35c4187b290e45baa255b68 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:50.760922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:50.762665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:50.778102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de2ff-469c-4cf1-9fe0-60567464aef4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:50.840695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:50.841298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:50.843080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:50.843355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] PATH_INFO: `/v3/users/046da16557e444cb9b4e518bab431b43` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:50.844697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=046da16557e444cb9b4e518bab431b43)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:50.850313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:51.050802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:51.123946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '046da16557e444cb9b4e518bab431b43'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.125154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '046da16557e444cb9b4e518bab431b43'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.138791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '046da16557e444cb9b4e518bab431b43'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.145794 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '046da16557e444cb9b4e518bab431b43'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.256110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '046da16557e444cb9b4e518bab431b43'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.257671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:51.264762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1139/2267] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:50 2022] DELETE /identity/v3/users/046da16557e444cb9b4e518bab431b43 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:51.549110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:51.554650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:51.567865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da203-3056-4ac7-aa0a-3859a7a9aca5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:51.628525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:51.629849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:51.629849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:51.630131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] PATH_INFO: `/v3/users/d4b43908501d4bc0951deb3463e67642` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:51.631277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b43908501d4bc0951deb3463e67642)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:51.633255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:51.779954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:51.781603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:51.797252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce902d2-c8eb-4278-be28-20e749f9d919 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:51.811360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:51.811360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:51.811360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:51.811360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] PATH_INFO: `/v3/projects/51b0543ac9f840b7a390a838f3557a37` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:51.812922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b0543ac9f840b7a390a838f3557a37)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:51.812922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:51.857974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b43908501d4bc0951deb3463e67642'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:51.867495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:52.011134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b0543ac9f840b7a390a838f3557a37', 'request_id': 'req-fde33f03-9356-4f56-87ba-761ff87955bf'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.020137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b43908501d4bc0951deb3463e67642'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.020137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b43908501d4bc0951deb3463e67642'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.020338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b0543ac9f840b7a390a838f3557a37', 'request_id': 'req-fde33f03-9356-4f56-87ba-761ff87955bf'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.031329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b0543ac9f840b7a390a838f3557a37', 'request_id': 'req-fde33f03-9356-4f56-87ba-761ff87955bf'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.031643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b43908501d4bc0951deb3463e67642'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.055813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-83581f38-5188-44f2-ac08-8711fda9014d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b43908501d4bc0951deb3463e67642'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.066994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1129/2268] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:51 2022] DELETE /identity/v3/users/d4b43908501d4bc0951deb3463e67642 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:52.068896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea36f8-3f1d-432e-bfde-3da0fa50f44b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.074273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea36f8-3f1d-432e-bfde-3da0fa50f44b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.074273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea36f8-3f1d-432e-bfde-3da0fa50f44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.097153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-47ea36f8-3f1d-432e-bfde-3da0fa50f44b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:52.098404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-47ea36f8-3f1d-432e-bfde-3da0fa50f44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:52.125083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] The token cache is being invalidate because project 51b0543ac9f840b7a390a838f3557a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:52.127959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.127959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-fde33f03-9356-4f56-87ba-761ff87955bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:52.132949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1140/2269] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:51 2022] DELETE /identity/v3/projects/51b0543ac9f840b7a390a838f3557a37 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:52.134699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.135193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.135555 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.167961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:52.169318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:52.177349 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1130/2270] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:52.180019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1141/2271] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:52.191047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.191047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.191047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.191420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.191420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:52.213940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:52.215206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:52.264740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1131/2272] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:52.268099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.273930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:52.280240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896871c-feb1-47ec-965e-187e591c6118 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.289719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.294504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.317955 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29d46b-4ff2-4c37-94e4-9c0ce57c1a25 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.324909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1132/2273] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:52.325968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.326136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.326615 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.326893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.327170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.342763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4affd5-a6b8-40fc-891d-2070cd01dfe1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.348065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.349567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.349933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.350279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.351245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.353383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.354613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.356123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.367884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1142/2274] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:52.496257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1133/2275] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:52.500481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.510767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697e205-1f96-43f9-94d7-689cbbaced82 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.581419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.581547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.581796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.582129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.584010 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.589713 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.688176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.700130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1143/2276] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:52.712136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a938a6-eed8-4920-b044-b6092f46858a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.729622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.729622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.729622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.729622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.730453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.730453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.750954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1134/2277] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:52.763274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.766870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064bb45-60e8-4140-a03b-00a3dffb3a2e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.771667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.772475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.772475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.772692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.773328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.774931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.799067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.799333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1144/2278] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:52.802403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1e21-b747-4e9a-ba9f-a94cd954dab5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.810444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.811237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.812230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.812580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] PATH_INFO: `/v3/projects/2b84f70e63d841b4bcb6b6309e2d1137/users/ffe40c42db71445bbcec98d6421ed0e7/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.830621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b84f70e63d841b4bcb6b6309e2d1137, user_id=ffe40c42db71445bbcec98d6421ed0e7, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.837799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b30c2-100e-4243-b08e-05c1d45f9292 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.886331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.890450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:52.909306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765ff6d-6be3-44aa-b118-207138333d22 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.915734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.916429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.917830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.917830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] PATH_INFO: `/v3/projects/e69948abdb664038ba01c95e87fcb4eb` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.918556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69948abdb664038ba01c95e87fcb4eb)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.920907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.931481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1135/2279] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:52 2022] PUT /identity/v3/projects/2b84f70e63d841b4bcb6b6309e2d1137/users/ffe40c42db71445bbcec98d6421ed0e7/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:52.942499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.943114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.943614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.983100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] MFA Rules not processed for user `ffe40c42db71445bbcec98d6421ed0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:52.985406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.063401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1136/2280] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:53.075212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.076827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.134185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780c6e2-9823-4629-b151-ce0436fa651b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.137889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.138550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.138828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.139103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.148400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.150054 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a79f7e-5a9f-4304-bdab-e58c3241dbd6 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.159894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1137/2281] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:22:53.189970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69948abdb664038ba01c95e87fcb4eb', 'request_id': 'req-d02caffd-3725-4fc1-9ef9-559e6e912019'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.195787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69948abdb664038ba01c95e87fcb4eb', 'request_id': 'req-d02caffd-3725-4fc1-9ef9-559e6e912019'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.206305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69948abdb664038ba01c95e87fcb4eb', 'request_id': 'req-d02caffd-3725-4fc1-9ef9-559e6e912019'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.291515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] The token cache is being invalidate because project e69948abdb664038ba01c95e87fcb4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:53.291774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.292466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d02caffd-3725-4fc1-9ef9-559e6e912019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.294458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1145/2282] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:52 2022] DELETE /identity/v3/projects/e69948abdb664038ba01c95e87fcb4eb => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:53.296702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:53.299544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.300024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.300131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.324573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:53.325867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.405951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1138/2283] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:53.413925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.414204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.414657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.446964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:53.448715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.457309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1146/2284] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:53.461715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.472130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f2dc5-7947-4f26-8ea6-f4fe66106cc6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.478487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.479116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.479420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.479993 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.485319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:53.485319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.485319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.503454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1139/2285] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:53.507907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.517431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808601e9-029d-4be9-9473-1e57e68e91ac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.522284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.522882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.523157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.523438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.524107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.526038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.659437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1147/2286] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:53 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:53.660559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.670478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53283efa-8031-4aba-9699-c0474214b81c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.730985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.731663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.731909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.732177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.732911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.735666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.891888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1140/2287] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:53 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:53.895735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.906785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3e7-1568-48c9-9495-04c60f83a0e7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.911461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.911984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.912275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.912548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.913146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.914747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.925363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1148/2288] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:53.929185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.945780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1eed9-9306-464d-9388-b8e22c326395 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.950637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.951250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.951526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.952072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.953166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.954779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7e966-b10e-40df-ac68-f35ddd99761e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.968122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1141/2289] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:53.972940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.002103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8424b6-549c-46d5-a830-3cd8e4e70fa4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.007725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.008564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.008834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.009111 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] PATH_INFO: `/v3/projects/91dae98b906f4228aa8175d68be4e4ef/users/d52d870e37644218bfff903f0f58c227/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.025895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=91dae98b906f4228aa8175d68be4e4ef, user_id=d52d870e37644218bfff903f0f58c227, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.028999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.150508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1149/2290] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:53 2022] PUT /identity/v3/projects/91dae98b906f4228aa8175d68be4e4ef/users/d52d870e37644218bfff903f0f58c227/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:54.162372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ba2f2-bf22-4ff1-a16d-e43edaed4b4c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.162943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ba2f2-bf22-4ff1-a16d-e43edaed4b4c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.163413 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ba2f2-bf22-4ff1-a16d-e43edaed4b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.195100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6f9ba2f2-bf22-4ff1-a16d-e43edaed4b4c None None] MFA Rules not processed for user `d52d870e37644218bfff903f0f58c227`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:54.196434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6f9ba2f2-bf22-4ff1-a16d-e43edaed4b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.277917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1142/2291] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:54.289694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.291450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.325922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:54.353873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c1a2c-d226-48c5-a6a4-5a8f5f8944cc admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.357640 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.358251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.358649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.358791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.367924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.369433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.379454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1150/2292] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:54.899538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:55.891317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:56.994850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cd043-e146-49f0-a5df-d1225c3e4a6e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:56.995728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cd043-e146-49f0-a5df-d1225c3e4a6e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:56.997738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cd043-e146-49f0-a5df-d1225c3e4a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:57.011039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:57.026609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2e2cd043-e146-49f0-a5df-d1225c3e4a6e None None] MFA Rules not processed for user `5d4092e9fbe24b2ca2f9c283cf22b68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:57.028285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2e2cd043-e146-49f0-a5df-d1225c3e4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:57.100799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1143/2293] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:57.155774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:57.393772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:57.925345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:58.581638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:58.609757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:59.320529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:59.322701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:59.400230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139e35f-2017-49b5-bf98-80b50895987f service glance] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:59.408408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:59.409817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:59.410105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:59.410632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] PATH_INFO: `/v3/users/4e93acdde18f4b30bd017512a113e16d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:59.411618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e93acdde18f4b30bd017512a113e16d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:59.418030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:59.489669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:59.500118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:59.500118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:59.516155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:59.517864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:59.602249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1144/2294] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:59.625885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:59.628378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:59.762036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e93acdde18f4b30bd017512a113e16d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:59.763385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e93acdde18f4b30bd017512a113e16d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:59.765036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa298e75-57a4-428a-8c9b-fd49b5ead306 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:59.771178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:59.772032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:59.772422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:59.772838 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:59.777319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:59.782046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e93acdde18f4b30bd017512a113e16d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:59.790668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e93acdde18f4b30bd017512a113e16d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:59.800268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:59.802436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e65105-d9ee-49db-a029-5af81f319704 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:59.829307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1145/2295] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:22:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:22:59.925891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:59.976781 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e93acdde18f4b30bd017512a113e16d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:59.987309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1151/2296] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:59 2022] DELETE /identity/v3/users/4e93acdde18f4b30bd017512a113e16d => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:59.991638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:59.992080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:59.992260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:00.015535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:00.016947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:00.102475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1146/2297] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:00.114779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:00.116808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:00.228818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0009f5-5230-4a6e-b660-2011946a4c79 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:00.234550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:00.235682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:00.236216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:00.236638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:00.250982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:00.252526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8d1ce-c7e7-4314-ad4f-461c27876013 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:00.271158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1152/2298] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:01.235571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:01.237623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:01.254294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0754ac-019c-4fd6-bdab-2633b1981f8a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:01.288429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:01.294765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:01.295718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:01.296594 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] PATH_INFO: `/v3/projects/ce37ac421e51434da853e6f5d8c2af4a` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:01.301071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce37ac421e51434da853e6f5d8c2af4a)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:01.303297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:01.613234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce37ac421e51434da853e6f5d8c2af4a', 'request_id': 'req-efdd02c2-e7b9-4314-98b0-e997c1dd4806'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:01.621357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce37ac421e51434da853e6f5d8c2af4a', 'request_id': 'req-efdd02c2-e7b9-4314-98b0-e997c1dd4806'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:01.757469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce37ac421e51434da853e6f5d8c2af4a', 'request_id': 'req-efdd02c2-e7b9-4314-98b0-e997c1dd4806'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:01.783581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] The token cache is being invalidate because project ce37ac421e51434da853e6f5d8c2af4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:23:01.785795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:01.785795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-efdd02c2-e7b9-4314-98b0-e997c1dd4806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:01.786789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1147/2299] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:23:01 2022] DELETE /identity/v3/projects/ce37ac421e51434da853e6f5d8c2af4a => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:01.792065 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:01.793280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:01.793715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:01.830358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:01.832990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:01.913752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1153/2300] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:01.922338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:01.922949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:01.923283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:01.953837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:01.953837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:01.991097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:01.996094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1148/2301] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:02.011663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3d39-1c6d-4047-a0af-e6da17a4114d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.013693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.015056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.015418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.015766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.016634 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:23:02.017048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.019254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.034219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1154/2302] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:23:02.039012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:02.050927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd6d3e-0969-4d4b-bd29-e53c6f333a9b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.056005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.056762 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.057053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.057318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.058128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.060083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.372744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1149/2303] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:23:02 2022] POST /identity/v3/projects => generated 383 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:02.378201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:02.389071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5548-6144-4e8c-823e-efc511b3a310 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.476436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.477105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.477449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.477783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.478500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.480561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.586069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:02.615637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1155/2304] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:23:02 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:23:02.616185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:02.641842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c8c74-d8f8-4b17-8e80-e37363fc8788 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.651577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.652797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.654184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.655968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.658889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.663314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.685416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1150/2305] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:23:02.695367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:02.704698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6185f-eb14-4be9-9d9c-387346463f1d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.710793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.711457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.711658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.711917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.712635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.714379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96875af-d9ba-41bd-bede-ee73913455b5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.726390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:02.731519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1156/2306] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:02.741849 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:02.754460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd6672-6861-413f-9671-acb94b5283b1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:02.760732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:02.761864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.762782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.762782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] PATH_INFO: `/v3/projects/466ff2cd26554461a0b96eaa9e553663/users/bbf35acbf20f4d02866411133bcd6443/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.791626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466ff2cd26554461a0b96eaa9e553663, user_id=bbf35acbf20f4d02866411133bcd6443, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:02.794595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:02.914018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1151/2307] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:02 2022] PUT /identity/v3/projects/466ff2cd26554461a0b96eaa9e553663/users/bbf35acbf20f4d02866411133bcd6443/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:02.926797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97076e85-7aaf-4713-856c-903f25bd34ee None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:02.927901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97076e85-7aaf-4713-856c-903f25bd34ee None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:02.928504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97076e85-7aaf-4713-856c-903f25bd34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:02.981538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-97076e85-7aaf-4713-856c-903f25bd34ee None None] MFA Rules not processed for user `bbf35acbf20f4d02866411133bcd6443`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:02.982953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-97076e85-7aaf-4713-856c-903f25bd34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:03.139928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1157/2308] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:03.181819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:03.185319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:03.317739 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed88f1-5856-483b-a3de-be369b621686 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:03.366233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:03.366233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:03.366233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:03.366628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:03.396482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:03.397604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:03.423273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1152/2309] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:05.935437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:06.344174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:08.659349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4024914-5ae4-4ab2-ae97-721256d46464 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:08.660116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4024914-5ae4-4ab2-ae97-721256d46464 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:08.660116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4024914-5ae4-4ab2-ae97-721256d46464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:08.681738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c4024914-5ae4-4ab2-ae97-721256d46464 None None] MFA Rules not processed for user `bd993c0493d447cb9b02e424a28e24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:08.683032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c4024914-5ae4-4ab2-ae97-721256d46464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:08.772199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1158/2310] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:08.991400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:09.485968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:10.861434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:10.940580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3712ba-2615-4798-939f-af42a713ccc4 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:10.946816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:10.948595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:10.948930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:10.949196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] PATH_INFO: `/v3/users/bbf35acbf20f4d02866411133bcd6443` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:10.950498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf35acbf20f4d02866411133bcd6443)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:10.953884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:11.786880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:11.804830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf35acbf20f4d02866411133bcd6443'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:11.999136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf35acbf20f4d02866411133bcd6443'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:12.005186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf35acbf20f4d02866411133bcd6443'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:12.010510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf35acbf20f4d02866411133bcd6443'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:12.032100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf35acbf20f4d02866411133bcd6443'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:12.042268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1153/2311] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:23:10 2022] DELETE /identity/v3/users/bbf35acbf20f4d02866411133bcd6443 => generated 0 bytes in 1182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:23:12.047151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:12.047655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:12.047754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:12.068559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:12.070190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:12.141622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1159/2312] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:12.148726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:12.150501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:12.233985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78648-a33f-47c0-80a9-703f7e0945b4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:12.237970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:12.238582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:12.238861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:12.239135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:12.255382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:12.266432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8fb62-74f4-4994-b3d9-733e4e7f93a6 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:12.268289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1154/2313] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:23:13.263255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:13.265244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:13.282506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fe3a3-3dfb-465d-9b70-cf922b4d367b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:13.303834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:13.303834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:13.303834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:13.303834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] PATH_INFO: `/v3/projects/466ff2cd26554461a0b96eaa9e553663` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:13.304190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466ff2cd26554461a0b96eaa9e553663)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:13.304190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:13.551433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466ff2cd26554461a0b96eaa9e553663', 'request_id': 'req-4e24d516-7a8e-4f87-9290-93c800d0b322'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:13.559638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466ff2cd26554461a0b96eaa9e553663', 'request_id': 'req-4e24d516-7a8e-4f87-9290-93c800d0b322'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:13.567680 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466ff2cd26554461a0b96eaa9e553663', 'request_id': 'req-4e24d516-7a8e-4f87-9290-93c800d0b322'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:13.800771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] The token cache is being invalidate because project 466ff2cd26554461a0b96eaa9e553663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:23:13.800989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:13.801654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4e24d516-7a8e-4f87-9290-93c800d0b322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:13.807028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1160/2314] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:23:13 2022] DELETE /identity/v3/projects/466ff2cd26554461a0b96eaa9e553663 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:13.809002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:13.809683 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:13.814885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:13.839373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:13.840861 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:13.941784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1155/2315] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:13.962848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:13.963126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:13.963563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.000710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:14.012150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:14.018319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1161/2316] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:14.019465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.033557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de49ed5-490c-4bac-abda-acbd46d77238 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.038440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.039107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.039379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.039648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.040340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:23:14.040638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.042617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.056552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1156/2317] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:23:14.060970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.071653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea4fcb-e5e1-4661-9d8c-f6872a87004a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.077124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.077728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.077999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.078270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.079083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.081110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.343153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1162/2318] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:23:14 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:14.347585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.359444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db46-faed-4473-8488-3080dfc5d199 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.419555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.420962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.421239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.421507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.422139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.424080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.722640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1157/2319] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:23:14 2022] POST /identity/v3/users => generated 287 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:14.726952 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.738696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb1ef7-7492-4f95-9bc9-2e0df5c9ef9c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.743411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.744843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.745189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.745507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.746277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.747998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.760190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1163/2320] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:14.765096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.776435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f0c48-403f-4c97-8a3d-612a92609d23 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.790129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.790129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.790129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.790129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.790935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.790935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d8acb-c5b2-46b9-9101-06335d90ae24 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.798473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1158/2321] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:14.802910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:14.813933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee5ee-7d1e-498d-ab84-65ffeab4b825 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:14.818681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:14.819290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:14.819571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:14.819998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] PATH_INFO: `/v3/projects/a0a3d5955e6545be83cdd6e2309f1adc/users/c5ed8fab49d24b8c9985b8978fc16c8f/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:14.835883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a3d5955e6545be83cdd6e2309f1adc, user_id=c5ed8fab49d24b8c9985b8978fc16c8f, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:14.840954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:14.995096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1164/2322] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:14 2022] PUT /identity/v3/projects/a0a3d5955e6545be83cdd6e2309f1adc/users/c5ed8fab49d24b8c9985b8978fc16c8f/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:15.006427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9fb64-7d7d-40ef-b247-be418b92d073 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:15.006909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9fb64-7d7d-40ef-b247-be418b92d073 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:15.007659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9fb64-7d7d-40ef-b247-be418b92d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:15.038690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a5c9fb64-7d7d-40ef-b247-be418b92d073 None None] MFA Rules not processed for user `c5ed8fab49d24b8c9985b8978fc16c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:15.040041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a5c9fb64-7d7d-40ef-b247-be418b92d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:15.107416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1159/2323] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:15.119443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:15.121172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:15.183911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d53ee-698a-4016-843c-fe6e32c137f0 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:15.187818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:15.188459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:15.188732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:15.189000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:15.199113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:15.200594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c221ad7-c778-4d0c-8a43-2ed009f4c7c9 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:15.212199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1165/2324] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:15.306431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:16.528045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:21.993471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:22.319368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:22.927043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:22.927417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:22.927920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:22.929287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1160/2325] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:23:22.936522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcefd7-5381-4ced-b245-32319c689458 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:22.936797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcefd7-5381-4ced-b245-32319c689458 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:22.937064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcefd7-5381-4ced-b245-32319c689458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:22.961458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-99fcefd7-5381-4ced-b245-32319c689458 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:22.966772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-99fcefd7-5381-4ced-b245-32319c689458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:23.060228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1166/2326] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:23:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:23.079308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:23.097164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:23.111792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:23.177976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25d344-1228-4760-9e65-711e40d38539 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:23.183467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:23.184136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.184460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.184744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.200556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:23.202102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07962d-21b0-4c35-b4f0-dc3465a3c160 service glance] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:23.214063 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1161/2327] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:23.486745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.487026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.487304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.488754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1167/2328] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:23:23.493777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d191-b9c0-420c-a175-38a7fdf02c45 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.494256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d191-b9c0-420c-a175-38a7fdf02c45 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.494566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d191-b9c0-420c-a175-38a7fdf02c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.514780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f418d191-b9c0-420c-a175-38a7fdf02c45 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:23.516133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f418d191-b9c0-420c-a175-38a7fdf02c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:23.570895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1162/2329] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:23:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:23:23.590112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:23.592569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:23.610681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f39e70-5fd7-4dc0-acc0-2540476e3b24 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:23.619760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:23.619760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.619760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.619760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.633017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:23.634598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3bb7c-0901-4625-a188-53e385a37494 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:23.642963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1168/2330] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:23.785756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c23e8-1c06-43b2-8e20-1a303e87267c None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.785756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c23e8-1c06-43b2-8e20-1a303e87267c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.785972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c23e8-1c06-43b2-8e20-1a303e87267c None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.787198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1163/2331] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:23:23.793386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.793670 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.793979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.815353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:23.816654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:23.827185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1169/2332] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:23:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:23.838604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c23e8-1c06-43b2-8e20-1a303e87267c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:23.850779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c23e8-1c06-43b2-8e20-1a303e87267c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:23.854827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:23.855523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:23.855822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:23.856167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:23.868593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:23.870102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da2a6a-db6d-49f8-ac8d-29830162d149 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:23.881958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1164/2333] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:23:24.505572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:24.689752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:25.787628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:25.865260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:26.519138 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:26.622183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:27.408974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:27.642540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:28.313377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:28.804128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:30.169501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:30.501910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:30.722116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:31.288394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:31.385334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ee6cf-8e2d-4ca3-9aa0-f54bd8748cac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:31.391053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:31.391809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:31.392659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:31.396298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] PATH_INFO: `/v3/users/c5ed8fab49d24b8c9985b8978fc16c8f` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:31.397923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ed8fab49d24b8c9985b8978fc16c8f)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:31.399923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:31.692121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ed8fab49d24b8c9985b8978fc16c8f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:31.692834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ed8fab49d24b8c9985b8978fc16c8f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:31.711888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:31.717222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ed8fab49d24b8c9985b8978fc16c8f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:31.721822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ed8fab49d24b8c9985b8978fc16c8f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:31.890211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ed8fab49d24b8c9985b8978fc16c8f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:31.897961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1170/2334] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:23:31 2022] DELETE /identity/v3/users/c5ed8fab49d24b8c9985b8978fc16c8f => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:31.902400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe553fc-4de6-454b-b00a-8ccea5086c73 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:31.902674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe553fc-4de6-454b-b00a-8ccea5086c73 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:31.903167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe553fc-4de6-454b-b00a-8ccea5086c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:31.944204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2fe553fc-4de6-454b-b00a-8ccea5086c73 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:31.945474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2fe553fc-4de6-454b-b00a-8ccea5086c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:32.049753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1165/2335] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:32.057248 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:32.058769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:32.129752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc2b7f-cee5-4296-8107-3c7fd3ce30c4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:32.133682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:32.134302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:32.134642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:32.134841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:32.146234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:32.147797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:32.165240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1171/2336] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:32.365521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:32.808899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:32.864996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:32.865562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:32.865657 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:32.902414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:32.904360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:32.905515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:32.907519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:32.932170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871c1-d02a-42a6-b81b-bcffd4cf8fae service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:32.939725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:32.940859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:32.941464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:32.942146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] PATH_INFO: `/v3/projects/a0a3d5955e6545be83cdd6e2309f1adc` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:32.943562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a3d5955e6545be83cdd6e2309f1adc)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:32.946604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:32.981072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1166/2337] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:32.997368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:32.999136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:33.093207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798eb88-22ec-4eea-8a69-d1804bc25d5c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:33.098326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:33.098891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.099166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.099438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.108384 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:33.110026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942183f4-71c0-437e-abc0-cc61939e6ab7 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:33.119124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1167/2338] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:23:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:23:33.171842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a3d5955e6545be83cdd6e2309f1adc', 'request_id': 'req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:33.177411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a3d5955e6545be83cdd6e2309f1adc', 'request_id': 'req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:33.183755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a3d5955e6545be83cdd6e2309f1adc', 'request_id': 'req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:33.324765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] The token cache is being invalidate because project a0a3d5955e6545be83cdd6e2309f1adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:23:33.326902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:33.328305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b43298b5-3af7-4bd7-bcd8-51d0b9bcceb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:33.332130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1172/2339] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:23:32 2022] DELETE /identity/v3/projects/a0a3d5955e6545be83cdd6e2309f1adc => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:23:33.337852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.338188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.338696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.384848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:33.387156 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:33.405966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:33.485816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1168/2340] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:23:33.496495 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.496970 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.497086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.533418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:33.534927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:33.557205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1173/2341] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:33.562440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:33.574596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7283c7c-07ad-43b0-a803-e10f475d0d6d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:33.580914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:33.581720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.582296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.582599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.583375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:23:33.584048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:33.585707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:33.598724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1169/2342] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:23:33.607484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:33.620317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b42b7d-9b37-4a43-bdd8-6e1d2e9254b8 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:33.627078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:33.627758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.628102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.628372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.629184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:33.643064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:33.756777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1174/2343] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:23:33 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:33.763333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:33.783500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6872d-843d-4371-a8f9-e11494aaee97 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:33.929703 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:33.930350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:33.930592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:33.930864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:33.932632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:33.939508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:33.959248 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:33.961118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.004912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10372-2b6b-42d8-bf9c-031addfecca2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.033432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.035927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.036559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.037218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] PATH_INFO: `/v3/users/d52d870e37644218bfff903f0f58c227` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.039512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52d870e37644218bfff903f0f58c227)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.042200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.108716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1170/2344] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:23:33 2022] POST /identity/v3/users => generated 286 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:34.112884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.128053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de9d05-3929-4946-8cf8-5b1276a93cfc admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.143228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.144214 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.144706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.149532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.150467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.152943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.168090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1171/2345] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:34.173725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.187443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce499a4f-f08d-45e3-a9a0-7215988321ed admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.193297 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.195708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.196248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.196719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.197565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.199382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.213656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1172/2346] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:34.222154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.241960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1f0b-fbfa-4a88-a7c8-05e5e88e3866 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.255216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.255216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.255216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.255216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] PATH_INFO: `/v3/projects/ce2ee7950a334c8888dae733cb698533/users/a111fd5f36324d75ac8ad55e762aeaca/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.269460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2ee7950a334c8888dae733cb698533, user_id=a111fd5f36324d75ac8ad55e762aeaca, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.272289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d0fd2-441d-4547-ad56-79b812bc0444 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.416962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1173/2347] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:34 2022] PUT /identity/v3/projects/ce2ee7950a334c8888dae733cb698533/users/a111fd5f36324d75ac8ad55e762aeaca/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:23:34.422009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.422337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.422579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.461198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:34.464776 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.468019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52d870e37644218bfff903f0f58c227'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.470838 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52d870e37644218bfff903f0f58c227'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.492760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52d870e37644218bfff903f0f58c227'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.515138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52d870e37644218bfff903f0f58c227'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.572074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1174/2348] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:34.583178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.588988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.597867 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7573d17b-cf18-432e-80c5-672cf0dbf30b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52d870e37644218bfff903f0f58c227'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.610741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1175/2349] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:23:33 2022] DELETE /identity/v3/users/d52d870e37644218bfff903f0f58c227 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:34.612529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.612807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.613490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.635728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:34.637086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.688172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511929-e782-4e6d-a4f5-af82b113d113 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.709229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1176/2350] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:34.718792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.720663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.766482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.767150 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.767737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.768190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.770052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.778629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:34.788181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.789807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.797005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e854b5b-95ec-467c-90d0-07344a03f5d0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.801995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.803092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.803734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1175/2351] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:34.816387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.824712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.839295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.841253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3142d-b4cf-436e-af9b-906a6133ab82 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.856164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1177/2352] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:23:35.828522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:36.029784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:36.031738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:36.048606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb588179-fdba-4e6d-b79d-d99c50d9fb5f service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:36.064482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:36.065622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.066062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.066340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] PATH_INFO: `/v3/projects/91dae98b906f4228aa8175d68be4e4ef` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.067682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=91dae98b906f4228aa8175d68be4e4ef)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:36.070319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:36.227026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91dae98b906f4228aa8175d68be4e4ef', 'request_id': 'req-2a02527b-ca4f-4f73-862b-b244db9fb1eb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:36.234227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91dae98b906f4228aa8175d68be4e4ef', 'request_id': 'req-2a02527b-ca4f-4f73-862b-b244db9fb1eb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:36.334690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91dae98b906f4228aa8175d68be4e4ef', 'request_id': 'req-2a02527b-ca4f-4f73-862b-b244db9fb1eb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:36.362767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] The token cache is being invalidate because project 91dae98b906f4228aa8175d68be4e4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:23:36.363161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:36.363937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2a02527b-ca4f-4f73-862b-b244db9fb1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:36.365823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1176/2353] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:23:36 2022] DELETE /identity/v3/projects/91dae98b906f4228aa8175d68be4e4ef => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:36.370749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.371304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.371304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.392331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:36.393693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:36.495534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1178/2354] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:36.502596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.506828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.506828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.554820 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:36.560103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:36.579945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1177/2355] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:36.586616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:36.629547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d8de6-5a17-40f4-a0d2-d5627694d3cd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:36.629547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:36.629547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.629547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.629999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.629999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:23:36.629999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:36.629999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:36.641020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1179/2356] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:23:36.642225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:36.651707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb435-b7b0-4f09-9782-dd9dd700cc82 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:36.664997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:36.666246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.666246 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.666479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.667124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:36.669493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:36.806457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1178/2357] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:23:36 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:36.810784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:36.825317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4645-3540-41f3-88c3-f89555a33356 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:36.913656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:36.918168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:36.918782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:36.919060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:36.920049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:36.920612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:36.922620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:37.067841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1180/2358] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:23:36 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:23:37.072061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:37.083125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215792ff-aac1-491e-baa1-bfdd1fd53bec admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:37.088113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:37.088725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:37.089045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:37.089342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:37.090000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:37.092494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:37.103458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1179/2359] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:37.107885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:37.118897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919dc4-a504-4a4f-816b-663f67f9acac admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:37.124263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:37.125890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:37.126384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:37.126829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:37.128105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:37.133267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0cee-174b-42be-902b-d0e10316f37e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:37.149144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:37.149888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1181/2360] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:37.159177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:37.166959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11691b5-1bd8-4e37-a8bc-4e2a89247084 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:37.180438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:37.181159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:37.181439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:37.181715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] PATH_INFO: `/v3/projects/0318904ace1f4263b64922161c127462/users/e47407ec9f4442cb96c59f6b83bc8998/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:37.199103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0318904ace1f4263b64922161c127462, user_id=e47407ec9f4442cb96c59f6b83bc8998, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:37.202735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:37.273615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1180/2361] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:37 2022] PUT /identity/v3/projects/0318904ace1f4263b64922161c127462/users/e47407ec9f4442cb96c59f6b83bc8998/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:23:37.289700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf29f3-15cc-4e39-a126-be4c63441515 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:37.290113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf29f3-15cc-4e39-a126-be4c63441515 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:37.290478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf29f3-15cc-4e39-a126-be4c63441515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:37.328775 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-87cf29f3-15cc-4e39-a126-be4c63441515 None None] MFA Rules not processed for user `e47407ec9f4442cb96c59f6b83bc8998`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:37.330597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-87cf29f3-15cc-4e39-a126-be4c63441515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:37.484056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1182/2362] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:37.505249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:37.505249 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:37.589526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee2e9-d552-4772-806c-7c89b1dc7325 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:37.600520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:37.601541 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:37.602031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:37.602509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:37.626282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:37.628406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:37.651466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1181/2363] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:37.754950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:37.957060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:38.942003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:39.371466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335dbcc-f4b8-4ea9-be45-3bcf0183c1a1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:39.372171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335dbcc-f4b8-4ea9-be45-3bcf0183c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:39.372278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335dbcc-f4b8-4ea9-be45-3bcf0183c1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:39.402474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:39.411093 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1335dbcc-f4b8-4ea9-be45-3bcf0183c1a1 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:39.415997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1335dbcc-f4b8-4ea9-be45-3bcf0183c1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:39.431116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:39.581517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1183/2364] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:39.639696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:39.641478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:39.791135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e15568-3a42-44f6-9c96-98641d0c79d7 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:39.797117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:39.797809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:39.798161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:39.798502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:39.808062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:39.810790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:39.821534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1182/2365] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:23:40.252460 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:41.185755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:42.781972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:43.049783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.039174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.137501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.246040 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:44.246793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:44.247995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:44.296750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] MFA Rules not processed for user `a111fd5f36324d75ac8ad55e762aeaca`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:44.298299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:44.301742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.390020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1184/2366] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:44.394702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:44.396174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:44.470091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6f04-09a6-4e99-802a-fd4340750d62 service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:44.475141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:44.475757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:44.476088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:44.476366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:44.477078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:44.478992 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:44.518545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.588229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1183/2367] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:23:44 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:23:44.592983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:44.603734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3827d82-70a8-4709-ac23-1b7e62b69617 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:44.649679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:44.659275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:44.660630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:44.660630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:44.660630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:44.661188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:44.663182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:44.769865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1185/2368] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:23:44 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:23:44.772791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:44.784842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b720a15-7c98-4936-a701-f990034aaaee admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:44.796708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:44.797545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:44.797946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:44.798317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:44.799072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:44.801618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:44.818184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1184/2369] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:44.819658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:44.833968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab433bf-aa7b-4f1c-95a2-27be648c8f48 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:44.845591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:44.846599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:44.847129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:44.847627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] PATH_INFO: `/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee/users/a73e5b749ea24be8b1c93d14a680b11d/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:44.873569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=535204ccbd4b4ad199b8f926f3bde6ee, user_id=a73e5b749ea24be8b1c93d14a680b11d, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:44.877795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:45.002658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1186/2370] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:44 2022] PUT /identity/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee/users/a73e5b749ea24be8b1c93d14a680b11d/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:45.008547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:45.022794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91632ad4-0cea-409e-a7f7-a48c22897d89 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:45.101653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:45.108862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:45.109525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:45.109848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:45.110231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:45.111355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:45.113020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fb73d-9601-49c3-8d82-5fc04a285400 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:45.141009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1185/2371] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:23:45.146131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:45.156429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:45.181801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef9291-1f76-42d2-8f92-24ea09bceebd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:45.197463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:45.206020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:45.209031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:45.209624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] PATH_INFO: `/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee/users/a73e5b749ea24be8b1c93d14a680b11d/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:45.217087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535204ccbd4b4ad199b8f926f3bde6ee, user_id=a73e5b749ea24be8b1c93d14a680b11d, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:45.220279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:45.241520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1187/2372] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:23:45 2022] PUT /identity/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee/users/a73e5b749ea24be8b1c93d14a680b11d/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:45.705843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:46.619472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:47.626448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:48.982044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:49.299536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:50.325915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:51.992618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:53.295360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:53.678600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:54.775834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:54.776408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:54.776408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:54.806767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] MFA Rules not processed for user `a73e5b749ea24be8b1c93d14a680b11d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:54.808049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:54.857123 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:54.882888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1186/2373] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:54.898384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:54.900193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:55.023053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea225f-cac5-48ec-8661-d715a3dea023 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:55.030201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:55.030633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:55.030930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:55.031094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:55.069464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:55.145562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:55.147361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:55.188096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1188/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:57.269556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:57.279372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:57.449831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd812-ce6b-4422-839f-e5e8e848e723 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:57.460601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:57.462222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:57.462510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:57.462800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:57.465227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:57.478483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:57.483006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970aa0d-1329-419b-bc72-bea5fbd6e568 service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:57.494785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1187/2375] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:58.078674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:58.079454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:58.301408 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75664-494a-42ba-a983-9da3bd2b9eb3 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:58.307073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:58.308338 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:58.309998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:58.310550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] PATH_INFO: `/v3/users/ffe40c42db71445bbcec98d6421ed0e7` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:58.312352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe40c42db71445bbcec98d6421ed0e7)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:58.314928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:58.544290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe40c42db71445bbcec98d6421ed0e7'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:58.544897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe40c42db71445bbcec98d6421ed0e7'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:58.573944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe40c42db71445bbcec98d6421ed0e7'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:58.604506 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe40c42db71445bbcec98d6421ed0e7'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:58.751757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe40c42db71445bbcec98d6421ed0e7'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:58.768331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1189/2376] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:23:58 2022] DELETE /identity/v3/users/ffe40c42db71445bbcec98d6421ed0e7 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:58.772720 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793493d-9e72-48b7-911f-21ea19f83636 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:58.774038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793493d-9e72-48b7-911f-21ea19f83636 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:58.774542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793493d-9e72-48b7-911f-21ea19f83636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:58.815409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c793493d-9e72-48b7-911f-21ea19f83636 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:58.816724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c793493d-9e72-48b7-911f-21ea19f83636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:59.050437 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1188/2377] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:59.061682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:59.063468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:59.165778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:59.170827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41367e-9487-40a5-9f8a-9392b09c5910 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:59.176750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:59.177928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:59.178374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:59.178758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:59.199096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:59.201012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:59.225692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1190/2378] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:00.076651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c253b2-307d-429b-97ed-8989fbfc6267 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.078284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c253b2-307d-429b-97ed-8989fbfc6267 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.078760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c253b2-307d-429b-97ed-8989fbfc6267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.102954 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:00.104818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:00.141318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fbb44-89f7-4c60-b818-7e30bb4c34a7 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:00.169560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:00.170899 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.171289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.171571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] PATH_INFO: `/v3/projects/2b84f70e63d841b4bcb6b6309e2d1137` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.175019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b84f70e63d841b4bcb6b6309e2d1137)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:00.183451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:00.184421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-55c253b2-307d-429b-97ed-8989fbfc6267 None None] MFA Rules not processed for user `a111fd5f36324d75ac8ad55e762aeaca`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:00.184421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-55c253b2-307d-429b-97ed-8989fbfc6267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:00.369510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1189/2379] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:00.443552 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b84f70e63d841b4bcb6b6309e2d1137', 'request_id': 'req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:00.459149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b84f70e63d841b4bcb6b6309e2d1137', 'request_id': 'req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:00.466719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b84f70e63d841b4bcb6b6309e2d1137', 'request_id': 'req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:00.558799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] The token cache is being invalidate because project 2b84f70e63d841b4bcb6b6309e2d1137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:24:00.559226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:00.560770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f979c76b-d8d2-4ff2-ad65-43e72d1dc9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:00.562725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1191/2380] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:24:00 2022] DELETE /identity/v3/projects/2b84f70e63d841b4bcb6b6309e2d1137 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:00.568335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.568624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.568922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.599728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:00.601447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:00.616635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:00.718873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1190/2381] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:00.790416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.790748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.790976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.817828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:00.819152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:00.849684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1192/2382] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:24:00.854234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:00.870889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7529f-7498-4f62-817b-281cd8f2f369 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:00.879575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:00.880261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.880725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.881023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.881714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:24:00.882022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:00.883694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:00.908132 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1191/2383] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:24:00.912316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:00.923513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba07a4-7ff8-4f02-bdef-71da67415bcc None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:00.928147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:00.928834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:00.929229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:00.929519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:00.930181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:00.932203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:01.076719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1193/2384] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:24:00 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:24:01.081596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:01.099029 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e084-00c0-4cd1-a9d4-f35e762c51a7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:01.183271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:01.183271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:01.183271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:01.183271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:01.184119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:01.186358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:01.317285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1192/2385] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:24:01 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:24:01.317933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:01.333588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bace59-995e-4b9d-84ba-51c22101dec2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:01.339828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:01.341019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:01.361301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:01.361301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:01.361301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:01.361301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:01.383948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1194/2386] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:24:01.384915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:01.411149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d7570-4e25-4f5b-85fb-ef5489bb47f2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:01.419511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:01.432553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:01.432882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:01.433161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:01.433925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:01.435688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d8b74-e630-4a3e-9c2d-9787d94bc958 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:01.459248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1193/2387] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:24:01.468022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:01.484853 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d3605-ba9d-45d3-a71d-06f337492e5f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:01.510275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:01.519208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:01.519208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:01.519208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] PATH_INFO: `/v3/projects/f19b2f57cc98428f86f0aa5f6a7693b0/users/6a3c2bfdce014429a74d3c3c61be1117/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:01.551058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19b2f57cc98428f86f0aa5f6a7693b0, user_id=6a3c2bfdce014429a74d3c3c61be1117, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:01.554665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:01.709748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1195/2388] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:24:01 2022] PUT /identity/v3/projects/f19b2f57cc98428f86f0aa5f6a7693b0/users/6a3c2bfdce014429a74d3c3c61be1117/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:01.722314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:01.722648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:01.722900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:01.809197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] MFA Rules not processed for user `6a3c2bfdce014429a74d3c3c61be1117`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:01.809197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:01.928083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1194/2389] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:01.943372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:01.943372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:02.052146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e82d2-90dc-4c59-a3ec-a1d4d8e12860 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:02.059291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:02.060263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:02.060548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:02.060824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:02.072447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:02.074020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca24637-e9a4-4465-9233-1a20dd3f1632 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:02.085844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1196/2390] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:02.956337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:02.958158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:03.006332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:03.114571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd0729-cbd1-4efc-aa7b-e7846c5abea7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:03.127127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:03.127794 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:03.128115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:03.128390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] PATH_INFO: `/v3/users/e47407ec9f4442cb96c59f6b83bc8998` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:03.129602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47407ec9f4442cb96c59f6b83bc8998)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:03.131956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:03.403989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47407ec9f4442cb96c59f6b83bc8998'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:03.530217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47407ec9f4442cb96c59f6b83bc8998'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:03.530862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47407ec9f4442cb96c59f6b83bc8998'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:03.540612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47407ec9f4442cb96c59f6b83bc8998'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:03.560901 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47407ec9f4442cb96c59f6b83bc8998'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:03.567884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1195/2391] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:24:02 2022] DELETE /identity/v3/users/e47407ec9f4442cb96c59f6b83bc8998 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:03.572568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:03.573922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:03.574269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:03.600421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:03.600421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:03.605951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:03.665976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1197/2392] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:03.676027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:03.680629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:03.741100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fe1d-1062-43de-a279-6d1ce3a4a5d4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:03.745285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:03.746858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:03.747391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:03.747533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:03.757840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:03.759397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bd455-bd43-4dc7-b342-28a65ae744f1 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:03.769884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1196/2393] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:24:04.462261 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:04.464997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:04.483622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acab82b-ce4b-47c7-9b3d-d2a29dc2eeb0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:04.495371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:04.497668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:04.498386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:04.499331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] PATH_INFO: `/v3/projects/0318904ace1f4263b64922161c127462` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:04.503729 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0318904ace1f4263b64922161c127462)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:04.526340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:04.740934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0318904ace1f4263b64922161c127462', 'request_id': 'req-c2f478a7-443c-487a-a1a7-0b50add0b62b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:04.749467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0318904ace1f4263b64922161c127462', 'request_id': 'req-c2f478a7-443c-487a-a1a7-0b50add0b62b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:04.760221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0318904ace1f4263b64922161c127462', 'request_id': 'req-c2f478a7-443c-487a-a1a7-0b50add0b62b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:04.853311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] The token cache is being invalidate because project 0318904ace1f4263b64922161c127462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:24:04.853731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:04.854931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:04.860519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1198/2394] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:24:04 2022] DELETE /identity/v3/projects/0318904ace1f4263b64922161c127462 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:24:11.313805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:11.314309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:11.314423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:11.339940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:11.341517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:11.383835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:11.410520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1197/2395] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:11.424553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:11.426033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:11.496942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f478a7-443c-487a-a1a7-0b50add0b62b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:11.502152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:11.502763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:11.503038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:11.503309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:11.511814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:11.513431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73741065-cb38-4db6-8e31-99abcc72f125 service swift] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:11.523452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1199/2396] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:24:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:24:11.868767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:11.870324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:11.953810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49521aec-5569-4b7b-b1b1-98b637efc90e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:11.958497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:11.959145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:11.959420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:11.959695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] PATH_INFO: `/v3/users/6a3c2bfdce014429a74d3c3c61be1117` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:11.960959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3c2bfdce014429a74d3c3c61be1117)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:11.962937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:12.284772 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3c2bfdce014429a74d3c3c61be1117'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:12.357756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3c2bfdce014429a74d3c3c61be1117'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:12.359120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3c2bfdce014429a74d3c3c61be1117'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:12.369285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3c2bfdce014429a74d3c3c61be1117'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:12.386652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3c2bfdce014429a74d3c3c61be1117'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:12.392844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1198/2397] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:24:11 2022] DELETE /identity/v3/users/6a3c2bfdce014429a74d3c3c61be1117 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:12.405402 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:12.405726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:12.406019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:12.426583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:12.427774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:12.447805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:12.493963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1200/2398] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:12.505286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:12.506752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:12.590607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257b8e-bff5-4a92-85fd-392a183115c6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:12.595467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:12.596109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:12.596387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:12.596659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:12.608364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:12.609878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c9df4-ff5d-496e-ac44-9a2d07b67329 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:12.620379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1199/2399] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:13.158667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:13.160168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:13.173289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95ec71-809a-4d42-befd-2b671605beac None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:13.179441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:13.180289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:13.180665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:13.181239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] PATH_INFO: `/v3/projects/f19b2f57cc98428f86f0aa5f6a7693b0` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:13.182520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19b2f57cc98428f86f0aa5f6a7693b0)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:13.185306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:13.351571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19b2f57cc98428f86f0aa5f6a7693b0', 'request_id': 'req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:13.357801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19b2f57cc98428f86f0aa5f6a7693b0', 'request_id': 'req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:13.364224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19b2f57cc98428f86f0aa5f6a7693b0', 'request_id': 'req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:13.454585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] The token cache is being invalidate because project f19b2f57cc98428f86f0aa5f6a7693b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:24:13.455185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:13.455684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:13.457246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1201/2400] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:24:13 2022] DELETE /identity/v3/projects/f19b2f57cc98428f86f0aa5f6a7693b0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:13.520651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:13.520651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:13.520651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:13.528837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:13.544122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:13.545072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:13.610761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1200/2401] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:24:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:13.625638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:13.628369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:13.718746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21d6d-b7b1-4cdd-9217-3611fff4f12c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:13.723984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:13.724597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:13.724876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:13.725148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:13.734894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:13.736797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:13.745465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1202/2402] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:24:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:24:17.102089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:17.103660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:17.174984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982769cb-6a2d-448e-8045-564c0f390343 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:17.180264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:17.180902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:17.181177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:17.181503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] PATH_INFO: `/v3/users/5d4092e9fbe24b2ca2f9c283cf22b68c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:17.182648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4092e9fbe24b2ca2f9c283cf22b68c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:17.185329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:17.412380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4092e9fbe24b2ca2f9c283cf22b68c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:17.477806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4092e9fbe24b2ca2f9c283cf22b68c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:17.478379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4092e9fbe24b2ca2f9c283cf22b68c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:17.487179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4092e9fbe24b2ca2f9c283cf22b68c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:17.500841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fb5b194e-2a06-4f8e-a406-c381ea261afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4092e9fbe24b2ca2f9c283cf22b68c'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:17.507316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1201/2403] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:24:17 2022] DELETE /identity/v3/users/5d4092e9fbe24b2ca2f9c283cf22b68c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:24:17.776637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:17.788798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0788c-cc02-49f3-a587-7ac7d98c6084 service swift] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:17.824818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:17.853877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:17.854526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:17.854808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:17.855079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] PATH_INFO: `/v3/projects/30da6d7509a64c8b904d8220993af210` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:17.856441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30da6d7509a64c8b904d8220993af210)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:17.858394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.005434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30da6d7509a64c8b904d8220993af210', 'request_id': 'req-6f93f4b1-a7e2-4277-be79-222abc30f47c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:18.012318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30da6d7509a64c8b904d8220993af210', 'request_id': 'req-6f93f4b1-a7e2-4277-be79-222abc30f47c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:18.018755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30da6d7509a64c8b904d8220993af210', 'request_id': 'req-6f93f4b1-a7e2-4277-be79-222abc30f47c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:18.112508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] The token cache is being invalidate because project 30da6d7509a64c8b904d8220993af210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:24:18.113034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:18.117992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-6f93f4b1-a7e2-4277-be79-222abc30f47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:24:18.117992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1203/2404] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:24:17 2022] DELETE /identity/v3/projects/30da6d7509a64c8b904d8220993af210 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:18.121916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.122396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.122660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.143816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:18.145140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:18.174584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:18.217624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1202/2405] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:24:18.225515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.225801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.226078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.247471 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:18.248708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:18.256624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1204/2406] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:18.261045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.270565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7e9f-ae8f-4aa7-9d6d-f276a386e974 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.275397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.276046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.276387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.276658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.277340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:24:18.277643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.279336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.291584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1203/2407] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:24:18.296477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.305976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42c8f7-21d5-40eb-a37c-a34868026d78 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.311371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.312018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.315395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.315395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.315395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.315395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.421331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1205/2408] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:24:18 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:24:18.430554 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.441753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad32e1-89b0-49d1-be62-be3f3126e3be admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.501726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.502329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.502606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.502880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.503508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.505479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.590418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1204/2409] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:24:18 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:24:18.595668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.607918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562e32-5a60-4ae0-886c-6698941d3fcb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.613996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.614773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.615595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.616806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.617643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.619742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.631280 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1206/2410] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:24:18.636039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.646824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c105-08b8-4925-8dae-984babda54e7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.651733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.652946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.653224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.653494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.654137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.655750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3d1b-e67d-4b74-a508-b61e8c6f431f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.671950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1205/2411] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:24:18.675084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.685502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82837237-d94a-4ae7-97c5-8590de3ea2a7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.690210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.690861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.691173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.691445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] PATH_INFO: `/v3/projects/fd9cf517497a40f6baebabe7bc3ea73f/users/1c994f449c0c45b3b04ba9c96d80a824/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.706596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9cf517497a40f6baebabe7bc3ea73f, user_id=1c994f449c0c45b3b04ba9c96d80a824, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:18.709311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:18.812653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1207/2412] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:24:18 2022] PUT /identity/v3/projects/fd9cf517497a40f6baebabe7bc3ea73f/users/1c994f449c0c45b3b04ba9c96d80a824/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:24:18.815337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90158-b891-43df-9666-a3925a26472d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.815650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90158-b891-43df-9666-a3925a26472d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.815956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90158-b891-43df-9666-a3925a26472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:18.835272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-05e90158-b891-43df-9666-a3925a26472d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:18.836673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-05e90158-b891-43df-9666-a3925a26472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:18.906516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1206/2413] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:18.916667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:18.919342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:18.993483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34cdf5-669b-4e78-9c10-d55bc681df07 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:18.997528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:18.998145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:18.998421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:18.998694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:19.008245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:19.009683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:19.020364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1208/2414] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:25.028965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.029491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.029589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.062789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] MFA Rules not processed for user `1c994f449c0c45b3b04ba9c96d80a824`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:25.064069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:25.095508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:25.147619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1207/2415] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:25.154071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.171621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bbcb7-ee13-4fac-99e2-fd1dc637268e service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.176389 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.177206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.177481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.177903 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.178562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.180539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.185250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:25.280015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1209/2416] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:24:25 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:24:25.284252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.295774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1820c-cc19-4875-91cb-4fd26ae8393f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.364344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.365083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.365362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.365633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.366285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.368364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.497474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1208/2417] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:24:25 2022] POST /identity/v3/users => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:24:25.501288 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.522166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b425587-944c-41fe-ae5c-b2f7d37d1fdd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.529330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.534298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.534298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.534298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.534298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.534298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.550267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1210/2418] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:24:25.554623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.566219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29024de-8a75-4a72-8d4d-68354cd05f1d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.571299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.574523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.574523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.574523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] PATH_INFO: `/v3/projects/dae001bf7162426b8865292ef9f50acd/users/c85f5356914347799fb064aae571fd56/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.589750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae001bf7162426b8865292ef9f50acd, user_id=c85f5356914347799fb064aae571fd56, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.592602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.691783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1209/2419] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:24:25 2022] PUT /identity/v3/projects/dae001bf7162426b8865292ef9f50acd/users/c85f5356914347799fb064aae571fd56/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:25.695927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.707382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7939c2-21d8-4922-9c5b-0bcd4a2655f7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.784421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.791427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.791427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.797918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.798741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.800559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27febc-1c64-44c0-9d06-28de49d963c9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.813912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1211/2420] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:24:25.818004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:25.830996 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cda4-8215-448a-8955-2eaa26fd728d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:25.835629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:25.836340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:25.842578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:25.842578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] PATH_INFO: `/v3/projects/dae001bf7162426b8865292ef9f50acd/users/c85f5356914347799fb064aae571fd56/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:25.842578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae001bf7162426b8865292ef9f50acd, user_id=c85f5356914347799fb064aae571fd56, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:25.842578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:25.863471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1210/2421] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:24:25 2022] PUT /identity/v3/projects/dae001bf7162426b8865292ef9f50acd/users/c85f5356914347799fb064aae571fd56/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:24:33.217915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca72-3f98-4957-a468-e756e4408c55 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:33.220444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca72-3f98-4957-a468-e756e4408c55 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:33.220444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca72-3f98-4957-a468-e756e4408c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:33.269572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-22c9ca72-3f98-4957-a468-e756e4408c55 None None] MFA Rules not processed for user `c85f5356914347799fb064aae571fd56`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:33.271087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-22c9ca72-3f98-4957-a468-e756e4408c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:33.364457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1212/2422] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:33.379312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:33.386738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:33.488587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ece27-6659-4e12-aace-20225d8ced45 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:33.492285 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:33.492902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:33.493181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:33.493457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:33.526057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:33.560939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:33.567717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:33.575940 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1211/2423] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:33.845314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a6f7-f606-4ebd-9250-0e5cc0b80519 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:33.845597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a6f7-f606-4ebd-9250-0e5cc0b80519 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:33.845898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a6f7-f606-4ebd-9250-0e5cc0b80519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:33.866623 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6353a6f7-f606-4ebd-9250-0e5cc0b80519 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:33.867828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6353a6f7-f606-4ebd-9250-0e5cc0b80519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:33.919384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:33.926825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1213/2424] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:24:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:24:33.939473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:33.941119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:34.029821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138157d2-ccfc-4b07-9df1-5126100fa76e service glance] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:34.035275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:34.035939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:34.036216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:34.036492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:34.045736 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:34.047474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:34.056552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1212/2425] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:24:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:24:44.052634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=46 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:44.924766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44ce1-d7eb-4675-aa46-6f50192bc9ce None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:44.925241 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44ce1-d7eb-4675-aa46-6f50192bc9ce None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:44.925352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44ce1-d7eb-4675-aa46-6f50192bc9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:44.945424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-91d44ce1-d7eb-4675-aa46-6f50192bc9ce None None] MFA Rules not processed for user `1c994f449c0c45b3b04ba9c96d80a824`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:44.946916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-91d44ce1-d7eb-4675-aa46-6f50192bc9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:44.954098 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1214/2426] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:47.421502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:47.476553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa3a61-945f-45e4-b1c5-85a0b56924cf service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:47.480747 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:47.481366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:47.481642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:47.481917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:47.483637 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:47.548930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:47.550888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:47.560566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1213/2427] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:24:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:24:49.350527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d90f8-b739-4ecc-a9f7-046832762b9d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:49.351060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d90f8-b739-4ecc-a9f7-046832762b9d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:49.351291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d90f8-b739-4ecc-a9f7-046832762b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:49.372705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-152d90f8-b739-4ecc-a9f7-046832762b9d None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:24:49.374084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-152d90f8-b739-4ecc-a9f7-046832762b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:24:49.381611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1215/2428] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:24:49.397421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:24:49.410924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057f947-3d68-4641-8db4-c2124e525875 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:24:49.416095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:24:49.417534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:24:49.417813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:24:49.418089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:24:49.426553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:24:49.428228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d553a-7d4c-481b-acc2-ec6f234b77a1 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:24:49.437010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1214/2429] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:24:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:24:59.366648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:24:59.434233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=56 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:11.614981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:11.615564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:11.615564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:11.635964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] MFA Rules not processed for user `74bfde91bf944824bdae8207641a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:25:11.637216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:11.699312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1216/2430] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:25:14.367948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:14.368558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:14.368558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:14.389082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] MFA Rules not processed for user `bd993c0493d447cb9b02e424a28e24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:25:14.390299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:14.452382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1215/2431] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:25:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:25:21.691406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=25 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:23.393852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:25:23.403683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35616801-58d2-4849-b1c8-2d79b3a56fbe None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:25:23.407578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:25:23.408308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:23.408650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:23.408989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:23.419839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:25:23.421380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:25:23.432030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1217/2432] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:25:24.444846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=25 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:33.426796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:36.195500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:25:36.207024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaec50-8ab6-4739-851e-7689545cf9f8 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:25:36.210661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:25:36.211271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:36.212264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:36.212603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:36.214357 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:36.290165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:25:36.291523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:25:36.302645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1216/2433] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:25:37.482005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:25:37.492005 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055107d2-5ae9-46d2-8c5a-b403cf99a0d1 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:25:37.496444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:25:37.497047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:37.497320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:37.497589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:37.499300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:37.513645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:25:37.515058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:25:37.525966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1218/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:25:46.297862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=34 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:47.520971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:00.539447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:00.541089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:00.600764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a996f45-7c7e-4f99-806b-23a5360ea58f service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:00.604321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:00.604922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:00.605200 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:00.605471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:00.607197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:00.619918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:00.621347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:00.630202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1217/2435] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:10.626212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=26 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:12.212422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:12.213826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:12.228224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bf6a-6674-402f-91ca-dde0ec2091ff service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:12.233878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:12.234483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:12.234762 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:12.235037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:12.236839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:12.248669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:12.250301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:12.259788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1219/2436] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:19.323587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:19.325511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:19.341656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e22ad8-766a-4c4c-a0c1-cb9d32799cf5 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:19.347988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:19.349136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:19.349421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:19.349692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:19.351580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:19.365178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:19.367325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd44c4c-8682-4eee-8921-c2ea8a0cb268 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:19.378859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1218/2437] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:22.255126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:27.768920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:27.770923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:27.783058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc875-7fbb-4637-88ee-321c6d756766 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:27.786800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:27.787985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:27.788258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:27.788526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:27.790260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:27.858646 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:27.862426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada78e5-3928-45ac-b58c-d514f44a73c0 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:27.871312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1220/2438] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:29.373929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=6 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:32.016542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:32.017908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:32.018402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:32.020295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1219/2439] 10.222.0.62 () {54 vars in 961 bytes} [Mon Aug 29 19:26:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:26:32.026225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:32.026751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:32.026751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:32.064578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] MFA Rules not processed for user `7bd6962ec6f5425091a001e8ee251616`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:32.066366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:32.183758 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1221/2440] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Aug 29 19:26:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:32.196488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:32.198656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:32.205856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:32.221982 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022b12-9ec8-40b5-97d0-168dd7112956 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:32.223360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9faaa1-8d42-4a40-95af-eb2a474076c7 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:32.229162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:32.229400 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:32.229972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:32.230204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:32.230314 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:32.230571 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] PATH_INFO: `/v3/users/bd993c0493d447cb9b02e424a28e24bc` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:32.230741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:32.231108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:32.231799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd993c0493d447cb9b02e424a28e24bc)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:32.233760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:32.241192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:32.251231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:32.253121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:32.269118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1222/2441] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:32.432903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd993c0493d447cb9b02e424a28e24bc'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:32.507475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd993c0493d447cb9b02e424a28e24bc'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:32.508192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd993c0493d447cb9b02e424a28e24bc'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:32.516286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd993c0493d447cb9b02e424a28e24bc'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:32.529444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-67076906-9e28-4cf3-9868-e45bc58c7fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd993c0493d447cb9b02e424a28e24bc'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:32.539914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1220/2442] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:26:32 2022] DELETE /identity/v3/users/bd993c0493d447cb9b02e424a28e24bc => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:26:32.817612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:32.850629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf30d4-7ac1-438c-b66a-cdf1d71499b6 service placement] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:32.948343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:32.948343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:32.948343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:32.948343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] PATH_INFO: `/v3/projects/fec9a23227784d23ba703719c785e413` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:32.948816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec9a23227784d23ba703719c785e413)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:32.948816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:32.976335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:33.065544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec9a23227784d23ba703719c785e413', 'request_id': 'req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:33.073725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec9a23227784d23ba703719c785e413', 'request_id': 'req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:33.082136 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec9a23227784d23ba703719c785e413', 'request_id': 'req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:33.202667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] The token cache is being invalidate because project fec9a23227784d23ba703719c785e413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:26:33.203091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:33.203843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-a915810b-b3db-4fd4-bc68-b228f2ac7ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:26:33.207451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1223/2443] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:26:32 2022] DELETE /identity/v3/projects/fec9a23227784d23ba703719c785e413 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:26:33.214079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2de35-911f-42ae-8a9b-ba0849b0065a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.215112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2de35-911f-42ae-8a9b-ba0849b0065a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.215708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2de35-911f-42ae-8a9b-ba0849b0065a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.237816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-57d2de35-911f-42ae-8a9b-ba0849b0065a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:33.239262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-57d2de35-911f-42ae-8a9b-ba0849b0065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:33.326390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1221/2444] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:33.334880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.340889 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.341358 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.347160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.347763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.348064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.374189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:33.385871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:33.395993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:33.399514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:33.424121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1224/2445] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:33.427535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.451474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd689414-3ea5-41aa-bc90-7950058eb1c7 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.455913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.456983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.457551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.458082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.459424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:26:33.460115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.463060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.479172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1225/2446] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:26:33.485474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.489998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1222/2447] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:26:33.496868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc370-7713-4349-b7b9-0d2561702b7d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.502843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.503530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.508054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.508054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.508054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.508054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.534610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.537874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:33.558368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:33.616575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d78eb-cff8-457f-9d7d-a963476d9e44 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.622108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1226/2448] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:26:33 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:26:33.622619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.623484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.623684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.624319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.626126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.635228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.637112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.650567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1223/2449] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:26:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:26:33.708249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3525233-99f7-4d31-9885-050f877b3dc4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.713474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.714176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.714519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.714851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.715715 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.717881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.813423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1227/2450] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:26:33 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:26:33.814620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.827248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3110766-5024-40ac-b211-41bf2d6d7b9c service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.835771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.838518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.838518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.838518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.838518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.839379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.855698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1224/2451] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:26:33.859966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.874949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5ff7-36c3-4cc1-8ca0-c0e7b2af803d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.884650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.885429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.885673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.885933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.886594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.888336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56239b5d-d1fd-4e5b-b290-08290c0e813a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:33.901949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1228/2452] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:26:33.910408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:33.924385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a2fd-a32d-46ed-ba14-f868b67a63e5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:33.934640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:33.935333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:33.935609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:33.935986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] PATH_INFO: `/v3/projects/68d8a00f546547178af95e8645c928e7/users/9c0c242c0ad048eb8fd35b4671704056/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:33.956672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d8a00f546547178af95e8645c928e7, user_id=9c0c242c0ad048eb8fd35b4671704056, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:33.966591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:34.070817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1225/2453] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:26:33 2022] PUT /identity/v3/projects/68d8a00f546547178af95e8645c928e7/users/9c0c242c0ad048eb8fd35b4671704056/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:26:34.075683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:34.076979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:34.077533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:34.101502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:34.103152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:34.161395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:34.195740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1229/2454] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:34.204621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:34.208632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:34.308370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45c0f7-7128-4c0a-8583-dcd3baa93e43 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:34.313605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:34.314484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:34.314969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:34.315445 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:34.338311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:34.348488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a100-ee16-4ec3-ab55-37f780f07c66 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:34.376615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1226/2455] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:34.400755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:34.402232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:34.552436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6ed96-04c4-4fa4-aa13-a84662e9ea64 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:34.569645 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:34.570205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:34.570504 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:34.570738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:34.572702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:34.605515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:34.607139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:34.627549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1230/2456] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:39.141821 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:39.143189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:39.143464 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:39.174443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] MFA Rules not processed for user `9d7e9a2846cc4ff3b64a8551a7a733a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:39.176126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:39.276248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1227/2457] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:39.748563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:39.750225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:39.770989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f629fc7-15df-4308-a473-a9eed603d3c2 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:39.778229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:39.779968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:39.780356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:39.781393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:39.783969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:39.804340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:39.806247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0043-7c07-41a4-bd84-64debd934422 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:39.819995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1231/2458] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:40.361338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:40.363807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:40.397695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:40.471332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef0168-9266-4577-b0c4-e80f3a0ae68d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:40.481380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:40.482325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:40.483606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:40.484141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:40.486578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:40.504039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:40.505850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:40.528968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1228/2459] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:42.027581 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:42.028573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:42.028829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:42.062223 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] MFA Rules not processed for user `9c0c242c0ad048eb8fd35b4671704056`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:42.063433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:42.148991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1232/2460] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:42.164074 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:42.189854 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f04e8-5dce-420a-a068-2cdf80d5f72e service placement] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:42.283070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:42.283070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:42.283070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:42.283070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:42.283818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:42.283818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90decfb4-dc5f-4824-9873-ffe4f4e31391 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:42.283818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1229/2461] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:43.205545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:43.246001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3891-eb0d-4bc3-b964-b16c5c3f920a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:43.257257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:43.258159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:43.258624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:43.259214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:43.261927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:43.279843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:43.282191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:43.296296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1233/2462] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:46.188050 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49882-2d38-4265-8569-8c04b82b38b1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:46.188934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49882-2d38-4265-8569-8c04b82b38b1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:46.189442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49882-2d38-4265-8569-8c04b82b38b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:46.219320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-33d49882-2d38-4265-8569-8c04b82b38b1 None None] MFA Rules not processed for user `9c0c242c0ad048eb8fd35b4671704056`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:26:46.233390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-33d49882-2d38-4265-8569-8c04b82b38b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:46.246527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1230/2463] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:26:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:26:47.827926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:26:47.831940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:47.869880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b9765-b8a0-497c-8673-5c74960fbb5b service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:26:47.886675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:26:47.887325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:26:47.887614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:26:47.888212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:26:47.893790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:26:47.923667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:26:47.925381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:26:47.954412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1234/2464] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:26:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:56.207104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=32 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:26:57.942262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:02.679479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e0e-27bc-4def-8d8a-4c3507769600 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:02.680411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e0e-27bc-4def-8d8a-4c3507769600 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:02.680755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e0e-27bc-4def-8d8a-4c3507769600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:02.726926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-46f28e0e-27bc-4def-8d8a-4c3507769600 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:27:02.728340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-46f28e0e-27bc-4def-8d8a-4c3507769600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:02.830002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1231/2465] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Aug 29 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:27:02.844359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:02.956045 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:02.956623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:02.957073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:03.003360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff12043-4408-4d30-92b6-66bf336b35e5 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:03.008161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:27:03.009902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:03.010050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:03.012599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:03.013313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:03.013828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:03.020417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1232/2466] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:27:03.025925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:03.027818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:03.036874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:03.045660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1235/2467] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:27:03.058640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e682-4b40-45cd-9a06-e61bc959db46 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:03.065163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:03.065875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:03.066259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:03.066572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:03.084043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:03.085999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:03.096024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1233/2468] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:27:13.040752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=50 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:13.091946 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=51 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:16.144101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:16.146457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:16.220576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974c35-08d8-4326-9eb8-a8481f162b8d service swift] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:16.226359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:16.227037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:16.227326 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:16.227597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] PATH_INFO: `/v3/users/1c994f449c0c45b3b04ba9c96d80a824` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:16.228993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c994f449c0c45b3b04ba9c96d80a824)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:16.230992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:16.452711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c994f449c0c45b3b04ba9c96d80a824'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.456512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c994f449c0c45b3b04ba9c96d80a824'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.465782 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c994f449c0c45b3b04ba9c96d80a824'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.472833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c994f449c0c45b3b04ba9c96d80a824'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.545934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c994f449c0c45b3b04ba9c96d80a824'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.554369 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1236/2469] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:27:16 2022] DELETE /identity/v3/users/1c994f449c0c45b3b04ba9c96d80a824 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:27:16.809164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:16.823253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d1b20-b9ac-4a10-b00b-93ee60246cbe service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:16.885223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:16.885975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:16.886255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:16.886529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] PATH_INFO: `/v3/projects/fd9cf517497a40f6baebabe7bc3ea73f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:16.887649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9cf517497a40f6baebabe7bc3ea73f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:16.890171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:16.987043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9cf517497a40f6baebabe7bc3ea73f', 'request_id': 'req-4a7e214d-da86-472c-911d-cfc529c8c7b4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:16.994615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9cf517497a40f6baebabe7bc3ea73f', 'request_id': 'req-4a7e214d-da86-472c-911d-cfc529c8c7b4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.069567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9cf517497a40f6baebabe7bc3ea73f', 'request_id': 'req-4a7e214d-da86-472c-911d-cfc529c8c7b4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.082809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] The token cache is being invalidate because project fd9cf517497a40f6baebabe7bc3ea73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:27:17.083096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.083748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.085299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1234/2470] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:27:16 2022] DELETE /identity/v3/projects/fd9cf517497a40f6baebabe7bc3ea73f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:27:17.089383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:17.091250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:17.105937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fad0f-de52-478b-97de-ebf82b0e54ab admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:17.167666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:17.169070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:17.169070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:17.169366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] PATH_INFO: `/v3/users/c85f5356914347799fb064aae571fd56` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:17.170545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85f5356914347799fb064aae571fd56)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:17.172499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:17.344207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:17.346735 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85f5356914347799fb064aae571fd56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.347519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85f5356914347799fb064aae571fd56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.366097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85f5356914347799fb064aae571fd56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.374505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85f5356914347799fb064aae571fd56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.444217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-f9cd6933-008c-4d5c-9f08-24646e4bb747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85f5356914347799fb064aae571fd56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:17.456560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1237/2471] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:27:17 2022] DELETE /identity/v3/users/c85f5356914347799fb064aae571fd56 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:27:17.791193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:17.792749 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:17.814015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e214d-da86-472c-911d-cfc529c8c7b4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:17.904204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:17.904836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:17.905140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:17.905406 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] PATH_INFO: `/v3/projects/dae001bf7162426b8865292ef9f50acd` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:17.906536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae001bf7162426b8865292ef9f50acd)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:17.908527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:18.043883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:18.058871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae001bf7162426b8865292ef9f50acd', 'request_id': 'req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:18.068353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae001bf7162426b8865292ef9f50acd', 'request_id': 'req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:18.157721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae001bf7162426b8865292ef9f50acd', 'request_id': 'req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:18.170706 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] The token cache is being invalidate because project dae001bf7162426b8865292ef9f50acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:27:18.170994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:18.171670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-be3f7a10-f9a5-4b53-8ac2-47f7dedb58c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:27:18.173441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1235/2472] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:27:17 2022] DELETE /identity/v3/projects/dae001bf7162426b8865292ef9f50acd => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:27:20.289509 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.290039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.290321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] PATH_INFO: `/` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.291705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1238/2473] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:27:20.297705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f40a-a266-4721-9a16-82e8ccfe84e8 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.298159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f40a-a266-4721-9a16-82e8ccfe84e8 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.298274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f40a-a266-4721-9a16-82e8ccfe84e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.342558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9f81f40a-a266-4721-9a16-82e8ccfe84e8 None None] MFA Rules not processed for user `94f1f1b96cd6477581e1040d8d8e0529`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:27:20.343774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9f81f40a-a266-4721-9a16-82e8ccfe84e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:20.407391 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1236/2474] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:27:20.420064 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:20.421801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:20.435454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3cbe-2c73-4520-b7ba-64bfc99bdfc6 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:20.439237 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:20.439842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.440163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.440463 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.448130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:20.449629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881da11-d221-4dea-b095-4dde46960f5d service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:20.457768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1239/2475] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:27:20.544281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.551879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.551879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] PATH_INFO: `/` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.551879 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1237/2476] 10.222.0.62 () {54 vars in 921 bytes} [Mon Aug 29 19:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:27:20.553755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9085-f33e-4882-bb85-1440f9dad9a5 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.554037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9085-f33e-4882-bb85-1440f9dad9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.554309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9085-f33e-4882-bb85-1440f9dad9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.576034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-829d9085-f33e-4882-bb85-1440f9dad9a5 None None] MFA Rules not processed for user `bf0af6b468c7442e818efec1543b7eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:27:20.581377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-829d9085-f33e-4882-bb85-1440f9dad9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:20.647984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1240/2477] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Aug 29 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:27:20.659670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:20.662118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:20.679333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079562-78c5-44f7-8e47-b7dc67cba58d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:20.683078 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:20.683685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:20.683995 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:20.684268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:20.695184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:20.696680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:20.709732 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1238/2478] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:27:22.385342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:22.385342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:22.385342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:22.404746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] MFA Rules not processed for user `b86144ce73784001bc405a8e2a4b1e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:27:22.405951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:22.459297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1241/2479] 10.222.0.62 () {58 vars in 967 bytes} [Mon Aug 29 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:27:22.471494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:22.473470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:22.543253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8d488-556b-4803-85cc-1c50bba5a46c service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:22.550466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd045b7cdd6024b6b82f3c4864c01e7d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:22.550756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:22.550756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:22.551369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:22.564550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:22.564550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:22.575131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1239/2480] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Aug 29 19:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:27:32.452239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=75,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:32.570295 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=86,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:27:43.457553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:27:43.459011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:43.476505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffe5bf-21f4-4afd-ba55-8033810bab8d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:27:43.485529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:27:43.485529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:27:43.485529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:27:43.485529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:27:43.487217 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:27:43.549740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:27:43.551346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11637212-6742-4477-ad83-3c77e3a6cc40 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:27:43.561722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1242/2481] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:27:53.556462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=28 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:16.082066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:16.092197 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2f5b-3916-4dba-9abf-fab0be74fdd5 service swift] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:16.096574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:16.097190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:16.097467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:16.097740 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:16.099453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:16.166137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:16.167834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:16.178618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1240/2482] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:19.543368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:19.543971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:19.544079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:19.574454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] MFA Rules not processed for user `9c0c242c0ad048eb8fd35b4671704056`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:19.576281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:19.644837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1243/2483] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:20.616748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:20.618784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:20.632950 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37661d06-0604-41b2-ada5-544047a648ff service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:20.638779 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:20.639595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:20.639900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:20.640175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] PATH_INFO: `/v3/users/74bfde91bf944824bdae8207641a08b7` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:20.641319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bfde91bf944824bdae8207641a08b7)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:20.644145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:20.894006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bfde91bf944824bdae8207641a08b7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.016837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bfde91bf944824bdae8207641a08b7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.017758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bfde91bf944824bdae8207641a08b7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.025833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bfde91bf944824bdae8207641a08b7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.039183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3793f267-8f7d-488d-a868-0d89bdac9736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bfde91bf944824bdae8207641a08b7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.045408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1241/2484] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:20 2022] DELETE /identity/v3/users/74bfde91bf944824bdae8207641a08b7 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:21.490384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:21.504037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f589c6-934a-492d-a5e9-90960bf62901 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:21.566770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:21.567370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:21.567876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:21.568194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] PATH_INFO: `/v3/projects/de427c8638c049d3ad809158b0ccc0d2` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:21.569337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de427c8638c049d3ad809158b0ccc0d2)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:21.571270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:21.763079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de427c8638c049d3ad809158b0ccc0d2', 'request_id': 'req-69b9d1b6-c821-421c-a113-4e414c58b587'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.769363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de427c8638c049d3ad809158b0ccc0d2', 'request_id': 'req-69b9d1b6-c821-421c-a113-4e414c58b587'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.775708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de427c8638c049d3ad809158b0ccc0d2', 'request_id': 'req-69b9d1b6-c821-421c-a113-4e414c58b587'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.876022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] The token cache is being invalidate because project de427c8638c049d3ad809158b0ccc0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:21.876310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.877469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-69b9d1b6-c821-421c-a113-4e414c58b587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:21.878988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1244/2485] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:21 2022] DELETE /identity/v3/projects/de427c8638c049d3ad809158b0ccc0d2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:21.883832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:21.884356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:21.884453 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.036668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:22.037926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:22.089056 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:22.097216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1242/2486] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:22.105057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.105340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.105614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.125804 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:22.127614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:22.135589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1245/2487] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:22.140095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.150433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afe4fd-014f-45a4-8811-8515209a8f0b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.155302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.155934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.156211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.156475 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.157110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:22.157403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.159506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.172914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1243/2488] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:28:22.179961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.190015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db83bdb-8438-4fdf-bd81-9a1fae0498dd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.194606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.195201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.195475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.195741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.196412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.198301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.344254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1246/2489] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:22 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:22.348573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.358509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe4eb2-f392-4ceb-9301-e7f961ab69d9 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.415442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.416418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.416698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.416965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.417577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.419550 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.559432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1244/2490] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:22 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:22.563514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.573886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4bccc-c522-4f1b-9c4b-22b27d6d1392 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.578458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.579052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.579330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.579669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.580499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.604232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.604232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:22.604232 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1247/2491] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:22.607028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.618287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1812c-e791-4c3b-b9b7-cd22ac7867b3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.622948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.623547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.623823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.624133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.624789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.626424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25cc45-f2eb-4354-9640-d6014b260a03 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.637429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1245/2492] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:22.641805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.651696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adbea1-bcb8-44bd-9ba3-226037472c09 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:22.661633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:22.661633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.661633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.661633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] PATH_INFO: `/v3/projects/cc2b8761a31d4d2eb67827bfd2686310/users/d9884a40c53a42b788f1de29fbac47b8/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.672783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc2b8761a31d4d2eb67827bfd2686310, user_id=d9884a40c53a42b788f1de29fbac47b8, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:22.676160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:22.814174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1248/2493] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:22 2022] PUT /identity/v3/projects/cc2b8761a31d4d2eb67827bfd2686310/users/d9884a40c53a42b788f1de29fbac47b8/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:22.824783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:22.825062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:22.825332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:22.852728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] MFA Rules not processed for user `d9884a40c53a42b788f1de29fbac47b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:22.854051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:22.918569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1246/2494] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:22.923330 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:22.996380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ad21-79b4-43ab-81cd-d4cd539d14ac admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.002395 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.003070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.003443 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.004049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.004386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.006553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.213519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1249/2495] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:22 2022] POST /identity/v3/projects => generated 363 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:23.223272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:23.229947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c47124-4f28-4276-8583-744ddd3feb05 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.287579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:23.294447 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.295110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.295386 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.295654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.296347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.299434 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.535462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1247/2496] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:23 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:23.539211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:23.550797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7960-126f-4578-a27d-8cdb436af88d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.555891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.556336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.556638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.556913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.557585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.559385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.584687 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1250/2497] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:23.589511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:23.601204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b91bce-5bfd-4dc9-9d87-4158cf6b84bb admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.606485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.607255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.607605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.607808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] PATH_INFO: `/v3/projects/7779439a2dca42f0b2eb94d6af888ed3/users/527119d337ed46d38d84264da5fcf860/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.624181 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7779439a2dca42f0b2eb94d6af888ed3, user_id=527119d337ed46d38d84264da5fcf860, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.626896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.728018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1248/2498] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:23 2022] PUT /identity/v3/projects/7779439a2dca42f0b2eb94d6af888ed3/users/527119d337ed46d38d84264da5fcf860/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:23.730267 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:23.756966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263ac9-ef2c-4bdb-b44c-a6cb27f88455 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.820693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.821308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.821583 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.821855 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.822695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.824924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7147d35-9d6c-4425-b731-43083d12d0f5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.837162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1251/2499] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:23.841539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:23.852998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbce96-2534-4cdf-bba1-8bd53fc7542f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:23.858144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:23.858880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.859195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.859477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] PATH_INFO: `/v3/projects/7779439a2dca42f0b2eb94d6af888ed3/users/527119d337ed46d38d84264da5fcf860/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.861552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7779439a2dca42f0b2eb94d6af888ed3, user_id=527119d337ed46d38d84264da5fcf860, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:23.864574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:23.875344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1249/2500] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:23 2022] PUT /identity/v3/projects/7779439a2dca42f0b2eb94d6af888ed3/users/527119d337ed46d38d84264da5fcf860/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:23.887986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:23.888282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:23.888542 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:23.920473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] MFA Rules not processed for user `527119d337ed46d38d84264da5fcf860`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:23.921768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:23.960834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:24.019514 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1252/2501] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:24.031609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:24.033665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:24.131761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17512591-36b9-4602-ad4d-f25a95c50a61 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:24.131761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:24.131761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:24.131761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:24.132415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:24.147829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:24.149572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b3fb-aca8-4e7e-955b-c74528200139 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:24.162577 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1250/2502] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:25.453563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:25.524760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab000-9d2a-490d-9f6c-a32c27805b62 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:25.530028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:25.530686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:25.531490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:25.531774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] PATH_INFO: `/v3/users/d9884a40c53a42b788f1de29fbac47b8` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:25.533505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9884a40c53a42b788f1de29fbac47b8)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:25.535898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:25.768818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9884a40c53a42b788f1de29fbac47b8'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:25.769849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9884a40c53a42b788f1de29fbac47b8'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:25.785992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9884a40c53a42b788f1de29fbac47b8'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:25.796158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9884a40c53a42b788f1de29fbac47b8'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:25.894260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9884a40c53a42b788f1de29fbac47b8'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:25.904533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1253/2503] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:25 2022] DELETE /identity/v3/users/d9884a40c53a42b788f1de29fbac47b8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:25.906288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:25.906765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:25.907198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:25.931401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:25.932986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:26.039426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1251/2504] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:26.053438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:26.055126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:26.113538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:26.118878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4546-6c52-46b3-ae97-f82f2c2e5001 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:26.122605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:26.123211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:26.123482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:26.123747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:26.132887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:26.134314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:26.144911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1254/2505] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:26.802308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:26.803744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:26.818184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b777-cd30-4b53-a759-2076493c9fab None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:26.832516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:26.833164 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:26.833439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:26.833708 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] PATH_INFO: `/v3/projects/cc2b8761a31d4d2eb67827bfd2686310` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:26.834922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc2b8761a31d4d2eb67827bfd2686310)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:26.837238 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:26.950657 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:26.967742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc2b8761a31d4d2eb67827bfd2686310', 'request_id': 'req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:26.973669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc2b8761a31d4d2eb67827bfd2686310', 'request_id': 'req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.042334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc2b8761a31d4d2eb67827bfd2686310', 'request_id': 'req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.055735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] The token cache is being invalidate because project cc2b8761a31d4d2eb67827bfd2686310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:27.059392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.059392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.059392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1252/2506] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:26 2022] DELETE /identity/v3/projects/cc2b8761a31d4d2eb67827bfd2686310 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:27.062462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:27.075031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd92b9-2c2c-41f3-b943-3d6386c55631 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:27.140892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:27.141529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:27.141834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:27.142659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] PATH_INFO: `/v3/users/527119d337ed46d38d84264da5fcf860` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:27.143960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=527119d337ed46d38d84264da5fcf860)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:27.145907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:27.394984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '527119d337ed46d38d84264da5fcf860'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.395727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '527119d337ed46d38d84264da5fcf860'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.407912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '527119d337ed46d38d84264da5fcf860'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.414860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '527119d337ed46d38d84264da5fcf860'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.547906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-e9c8eebc-80af-499c-923b-d72e8564f145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '527119d337ed46d38d84264da5fcf860'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:27.555245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1255/2507] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:27 2022] DELETE /identity/v3/users/527119d337ed46d38d84264da5fcf860 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:28.138484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:28.140103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:28.155215 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61fa2d-894f-4cfd-9a2a-54fb5b3a5801 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:28.217189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:28.218006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.218289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.218561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] PATH_INFO: `/v3/projects/7779439a2dca42f0b2eb94d6af888ed3` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.219719 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7779439a2dca42f0b2eb94d6af888ed3)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:28.222277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:28.374807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7779439a2dca42f0b2eb94d6af888ed3', 'request_id': 'req-f683a939-eab7-4b7c-b3b5-a847194948f3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:28.380317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7779439a2dca42f0b2eb94d6af888ed3', 'request_id': 'req-f683a939-eab7-4b7c-b3b5-a847194948f3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:28.458670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7779439a2dca42f0b2eb94d6af888ed3', 'request_id': 'req-f683a939-eab7-4b7c-b3b5-a847194948f3'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:28.471781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] The token cache is being invalidate because project 7779439a2dca42f0b2eb94d6af888ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:28.472160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:28.472813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f683a939-eab7-4b7c-b3b5-a847194948f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:28.474283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1253/2508] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:28 2022] DELETE /identity/v3/projects/7779439a2dca42f0b2eb94d6af888ed3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:28.479361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.479942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.479942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.500361 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:28.502060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:28.561818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1256/2509] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:28.573380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.573657 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.573959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.594332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:28.595537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:28.603480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1254/2510] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:28.607797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:28.617994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a157e-2c14-4d10-8555-964c6a4dc1cb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:28.622576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:28.623170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.623448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.623718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.624398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:28.624700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:28.626830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:28.629352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:28.638797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1257/2511] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:28:28.643870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:28.654757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e83e3b-82f6-442a-8291-5592b591e5f0 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:28.659564 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:28.660321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.660546 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.660814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.661458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:28.663431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:28.801819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1255/2512] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:28 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:28.805711 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:28.816450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb4dc-b482-4a53-91e1-0373e170fa0d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:28.874137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:28.874746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:28.875023 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:28.875295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:28.875957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:28.878436 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:28.995643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1258/2513] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:28 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:28.999876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.012279 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170a75-b899-4c8a-8c93-a0d42203bf03 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.018118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.018769 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.022061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.022061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.022061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.022061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.032566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1256/2514] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:29.036677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.047122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8a648-6567-4f98-bb4f-3ec39bf65d27 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.053229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.053878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.054159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.054431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.055068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.056720 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65974b4-f83e-45b7-8a57-6dffe86ffe1c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.068488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1259/2515] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:29.072755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.079976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:29.089222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101317-59b3-4050-b90b-5ab2efee3a29 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.089408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.090113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.090383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.090652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] PATH_INFO: `/v3/projects/e49527b9964b4e0689058c849391564a/users/df0dba891c894855b15a05c6c230726f/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.105929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49527b9964b4e0689058c849391564a, user_id=df0dba891c894855b15a05c6c230726f, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.108893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.229461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1257/2516] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:29 2022] PUT /identity/v3/projects/e49527b9964b4e0689058c849391564a/users/df0dba891c894855b15a05c6c230726f/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:29.240737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.241017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.241313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.269666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] MFA Rules not processed for user `df0dba891c894855b15a05c6c230726f`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:29.270815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:29.336859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1260/2517] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:29.341339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.408158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696103d-9fb6-42f4-9388-31f8e8b53c68 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.412836 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.413431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.413707 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.413975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.414685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.416620 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.533313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1258/2518] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:29 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:29.537430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.547784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88286571-c448-4dbe-8c99-a2808dfbae89 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.607953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.608576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.608872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.609129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.609763 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.612032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.622570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:29.745378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1261/2519] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:29 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:29.749507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.761394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082ead-d42f-4116-8996-ff8a4057b0eb admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.767463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.768673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.769014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.769286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.769927 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.779768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.783583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1259/2520] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:29.787819 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.799601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85874d-1717-42f3-ac19-4d2073b6547d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.804475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.805144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.805446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.805754 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] PATH_INFO: `/v3/projects/4ec22d042b064fdea32f40919c48ea66/users/f2c8446e2ad7409dbfffec416c38b6d1/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.821893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec22d042b064fdea32f40919c48ea66, user_id=f2c8446e2ad7409dbfffec416c38b6d1, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.824697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:29.902235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1262/2521] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:29 2022] PUT /identity/v3/projects/4ec22d042b064fdea32f40919c48ea66/users/f2c8446e2ad7409dbfffec416c38b6d1/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:29.906282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:29.916374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268460bd-cde5-44e1-ab9a-595a0a3f66bf admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:29.982443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:29.983184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:29.983461 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:29.983714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:29.984415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:29.986632 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca950fdd-65b7-45c6-a13f-eda511be36c8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:30.000260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1260/2522] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:30.005439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:30.015997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc161e6-b63c-4aa3-9f07-13b8d3c32337 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:30.020963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:30.021629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.021909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.022182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] PATH_INFO: `/v3/projects/4ec22d042b064fdea32f40919c48ea66/users/f2c8446e2ad7409dbfffec416c38b6d1/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.024414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec22d042b064fdea32f40919c48ea66, user_id=f2c8446e2ad7409dbfffec416c38b6d1, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:30.027073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:30.038525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1263/2523] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:30 2022] PUT /identity/v3/projects/4ec22d042b064fdea32f40919c48ea66/users/f2c8446e2ad7409dbfffec416c38b6d1/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:30.050871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.051149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.051418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.080704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] MFA Rules not processed for user `f2c8446e2ad7409dbfffec416c38b6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:30.081908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:30.129134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:30.158476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1261/2524] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:30.184467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:30.184467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:30.237228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee843f-8664-44c1-ac09-c4a2d958d981 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:30.241560 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:30.242171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.244961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.244961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.255985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:30.257524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddef925-e335-49e9-a26f-6b382b4f73fe service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:30.267286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1264/2525] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:30.297697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:30.359774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22b9b4-7f8a-49e5-a50d-a6e07595cabd None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:30.365179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:30.365834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.366107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.366375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] PATH_INFO: `/v3/users/df0dba891c894855b15a05c6c230726f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.367543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0dba891c894855b15a05c6c230726f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:30.369535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:30.642416 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0dba891c894855b15a05c6c230726f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:30.719038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0dba891c894855b15a05c6c230726f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:30.719666 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0dba891c894855b15a05c6c230726f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:30.727810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0dba891c894855b15a05c6c230726f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:30.742804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0dba891c894855b15a05c6c230726f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:30.749366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1262/2526] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:30 2022] DELETE /identity/v3/users/df0dba891c894855b15a05c6c230726f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:30.754876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.762766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.762766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.776162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:30.777515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:30.846816 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1265/2527] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:30.856695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:30.862131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:30.919806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13fa0-10d7-4902-ac8b-f9f8f8258dd3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:30.923624 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:30.924277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:30.924553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:30.924823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:30.934189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:30.935629 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:30.940148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:30.946504 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1263/2528] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:31.500525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:31.503336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:31.517222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb75d0-509a-40f9-a748-a650ccf00dc0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:31.522335 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:31.522981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:31.525705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:31.525705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] PATH_INFO: `/v3/projects/e49527b9964b4e0689058c849391564a` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:31.525705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49527b9964b4e0689058c849391564a)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:31.526872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:31.713007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49527b9964b4e0689058c849391564a', 'request_id': 'req-5522da4d-a817-4aae-bb0c-c8289ad0ab02'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:31.717260 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:31.720520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49527b9964b4e0689058c849391564a', 'request_id': 'req-5522da4d-a817-4aae-bb0c-c8289ad0ab02'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:31.727423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49527b9964b4e0689058c849391564a', 'request_id': 'req-5522da4d-a817-4aae-bb0c-c8289ad0ab02'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:31.858608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] The token cache is being invalidate because project e49527b9964b4e0689058c849391564a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:31.859275 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:31.859590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:31.861817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1266/2529] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:31 2022] DELETE /identity/v3/projects/e49527b9964b4e0689058c849391564a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:31.866144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:31.875751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbea8-4b9b-4d0d-983c-12d04d1c6972 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:31.932304 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:31.932764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:31.933037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:31.933316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] PATH_INFO: `/v3/users/f2c8446e2ad7409dbfffec416c38b6d1` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:31.934440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c8446e2ad7409dbfffec416c38b6d1)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:31.936393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:32.204578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c8446e2ad7409dbfffec416c38b6d1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:32.420082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c8446e2ad7409dbfffec416c38b6d1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:32.420768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c8446e2ad7409dbfffec416c38b6d1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:32.447212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c8446e2ad7409dbfffec416c38b6d1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:32.448484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c8446e2ad7409dbfffec416c38b6d1'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:32.457121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1264/2530] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:31 2022] DELETE /identity/v3/users/f2c8446e2ad7409dbfffec416c38b6d1 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:32.503304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:32.504849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:32.572839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522da4d-a817-4aae-bb0c-c8289ad0ab02 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:32.622986 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:32.623603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:32.623936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:32.624211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:32.626704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:32.691911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:32.693352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f30c5b-58cf-4754-8765-39063ad7c4e6 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:32.703593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1267/2531] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:33.122598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:33.124673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:33.147615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373bd2b-28d5-4c1a-8dcb-af7a0e9df0cf admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:33.148765 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:33.149643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.150099 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.150364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] PATH_INFO: `/v3/projects/4ec22d042b064fdea32f40919c48ea66` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.151540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec22d042b064fdea32f40919c48ea66)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:33.153520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:33.357141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec22d042b064fdea32f40919c48ea66', 'request_id': 'req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:33.364712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec22d042b064fdea32f40919c48ea66', 'request_id': 'req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:33.447806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec22d042b064fdea32f40919c48ea66', 'request_id': 'req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:33.457850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:33.461630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] The token cache is being invalidate because project 4ec22d042b064fdea32f40919c48ea66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:33.461941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:33.462604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-a6ee16bc-5a03-4d8c-98af-293f7e32cce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:33.464300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1265/2532] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:33 2022] DELETE /identity/v3/projects/4ec22d042b064fdea32f40919c48ea66 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:33.469482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.470598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.470598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.492214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:33.493447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:33.557949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1268/2533] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:33.566052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.566355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.566598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.588008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:33.589302 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:33.598135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1266/2534] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:33.602367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:33.612390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed02328-b3e6-4d61-a4cb-f6b166f5f7f9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:33.617404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:33.618053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.618328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.618599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.619263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:33.619563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:33.621283 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:33.625419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:33.635622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1269/2535] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:28:33.640253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:33.650790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac8cc-91d1-442c-9cdd-d67c59e4330b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:33.655487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:33.656141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.656415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.656685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.657917 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:33.659907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:33.804976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1267/2536] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:33 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:33.805953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:33.816029 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e59a1-f0fc-4c77-b0db-556ccbaf67a6 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:33.875437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:33.876112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:33.876392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:33.876663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:33.877316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:33.879277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.052334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1270/2537] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:33 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:34.053713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.064316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b926-6e86-434f-8f18-5aa2573cfe99 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.069312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.069907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.070185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.070535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.071171 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.072932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.083168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1268/2538] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:34.087357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.097882 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcbf85-c020-42e8-a9e7-7bd7d3a76dc5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.102549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.103142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.103416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.103685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.104420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.106021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896fbc-0bc4-4e9f-ab36-69989cb1b28e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.117268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1271/2539] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:34.121729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.131578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004b61-58a1-4526-96ba-df2de2b9a0be admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.136227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.136864 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.137136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.137405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] PATH_INFO: `/v3/projects/ba18cb8a7c264cc4832db09b35755e93/users/558eea2795f24acf92f9fba284f3d389/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.152137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba18cb8a7c264cc4832db09b35755e93, user_id=558eea2795f24acf92f9fba284f3d389, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.154785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.282237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1269/2540] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:34 2022] PUT /identity/v3/projects/ba18cb8a7c264cc4832db09b35755e93/users/558eea2795f24acf92f9fba284f3d389/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:34.294400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.294682 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.294953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.331337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] MFA Rules not processed for user `558eea2795f24acf92f9fba284f3d389`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:34.333025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:34.412008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1272/2541] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:34.412466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.483646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c482-54c7-4783-817d-c5a66f10150d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.489189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.489796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.490076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.490347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.490975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.492929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.659615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1270/2542] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:34 2022] POST /identity/v3/projects => generated 389 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:34.663683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.680457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fab92a-d950-49fa-b964-e31e94876f5a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.739478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.740126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.740406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.740677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.741325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.743250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.773279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:34.850728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1273/2543] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:34 2022] POST /identity/v3/users => generated 301 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:34.854755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.866539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3976-4999-4ea5-83ee-a8f1eaf5c71a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.871825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.872573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.872910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.873250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.874101 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.876398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:34.889547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1271/2544] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:34.894343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:34.905169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb0a8-3037-4b8f-9ffe-a9fee50be493 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:34.912025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:34.913446 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:34.914301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:34.914683 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] PATH_INFO: `/v3/projects/b2277c37a0674476ba2d1dd1b59125c8/users/c26a48a3018d44488af3db6ba46e1f38/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:34.933166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2277c37a0674476ba2d1dd1b59125c8, user_id=c26a48a3018d44488af3db6ba46e1f38, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:34.936386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:35.026475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1274/2545] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:34 2022] PUT /identity/v3/projects/b2277c37a0674476ba2d1dd1b59125c8/users/c26a48a3018d44488af3db6ba46e1f38/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:35.029524 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:35.037910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe859-e9ba-4546-a4fc-842cd74f0e71 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:35.112712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:35.113582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:35.113925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:35.114119 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:35.114858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:35.116533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891f8bd-96ab-476c-9d48-2be94ed06d6f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:35.131147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1272/2546] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:35.132547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:35.143647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871217a-27b8-4005-b93b-397547908bee admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:35.148910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:35.149744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:35.150089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:35.150476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] PATH_INFO: `/v3/projects/b2277c37a0674476ba2d1dd1b59125c8/users/c26a48a3018d44488af3db6ba46e1f38/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:35.152778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2277c37a0674476ba2d1dd1b59125c8, user_id=c26a48a3018d44488af3db6ba46e1f38, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:35.155603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:35.168213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1275/2547] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:35 2022] PUT /identity/v3/projects/b2277c37a0674476ba2d1dd1b59125c8/users/c26a48a3018d44488af3db6ba46e1f38/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:35.179152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:35.179680 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:35.179918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:35.197897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:35.216713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] MFA Rules not processed for user `c26a48a3018d44488af3db6ba46e1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:35.218107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:35.299916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1273/2548] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:35.315992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:35.315992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:35.394620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5ccf-8fd0-4ffe-8e90-087f07dc4e8c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:35.398285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:35.398891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:35.399164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:35.399489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:35.411153 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:35.413224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a5ce9-7ebb-4da0-879e-eebdf0d4d0fa service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:35.425526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1276/2549] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:35.587440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:35.651625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6c186-e030-4e1c-b944-65a68d0a7c56 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:35.656272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:35.656902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:35.657240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:35.657508 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] PATH_INFO: `/v3/users/558eea2795f24acf92f9fba284f3d389` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:35.659208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558eea2795f24acf92f9fba284f3d389)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:35.660568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:35.952905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558eea2795f24acf92f9fba284f3d389'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:36.136448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558eea2795f24acf92f9fba284f3d389'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:36.137112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558eea2795f24acf92f9fba284f3d389'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:36.145284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558eea2795f24acf92f9fba284f3d389'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:36.170365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558eea2795f24acf92f9fba284f3d389'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:36.181064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1274/2550] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:35 2022] DELETE /identity/v3/users/558eea2795f24acf92f9fba284f3d389 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:36.185814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:36.186212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:36.186212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:36.225209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:36.231054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:36.299743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1277/2551] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:36.311967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:36.314051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:36.336383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:36.390867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b56f4-a0ed-4b25-af1c-942dc9877706 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:36.395231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:36.395884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:36.396332 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:36.396607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:36.411962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:36.411962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:36.422885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1275/2552] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:37.290884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:37.291956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:37.305562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea01503-4240-4bdc-baa0-0d65ca0378e4 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:37.311269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:37.312561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:37.312840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:37.313114 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] PATH_INFO: `/v3/projects/ba18cb8a7c264cc4832db09b35755e93` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:37.315225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba18cb8a7c264cc4832db09b35755e93)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:37.317393 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:37.583628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:37.594060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba18cb8a7c264cc4832db09b35755e93', 'request_id': 'req-991986c0-7321-4243-9bc5-b98d9087ad75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:37.609926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:37.612008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:37.624470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba18cb8a7c264cc4832db09b35755e93', 'request_id': 'req-991986c0-7321-4243-9bc5-b98d9087ad75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:37.636187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba4ebb-aeae-498e-b231-3d47be1717de service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:37.639250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba18cb8a7c264cc4832db09b35755e93', 'request_id': 'req-991986c0-7321-4243-9bc5-b98d9087ad75'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:37.702916 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:37.703559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:37.703846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:37.704808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] PATH_INFO: `/v3/users/9d7e9a2846cc4ff3b64a8551a7a733a2` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:37.705990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7e9a2846cc4ff3b64a8551a7a733a2)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:37.707963 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:37.761826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] The token cache is being invalidate because project ba18cb8a7c264cc4832db09b35755e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:37.762119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:37.762741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:37.764588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1278/2553] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:37 2022] DELETE /identity/v3/projects/ba18cb8a7c264cc4832db09b35755e93 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:37.768553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:37.769868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:37.783365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991986c0-7321-4243-9bc5-b98d9087ad75 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:37.788180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:37.788831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:37.789107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:37.789377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] PATH_INFO: `/v3/users/c26a48a3018d44488af3db6ba46e1f38` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:37.790751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26a48a3018d44488af3db6ba46e1f38)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:37.792931 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:38.007537 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7e9a2846cc4ff3b64a8551a7a733a2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.052922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26a48a3018d44488af3db6ba46e1f38'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.053724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26a48a3018d44488af3db6ba46e1f38'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.073175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26a48a3018d44488af3db6ba46e1f38'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.081104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26a48a3018d44488af3db6ba46e1f38'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.125431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7e9a2846cc4ff3b64a8551a7a733a2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.130354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7e9a2846cc4ff3b64a8551a7a733a2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.139601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7e9a2846cc4ff3b64a8551a7a733a2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.156422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7e9a2846cc4ff3b64a8551a7a733a2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.163956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1276/2554] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:37 2022] DELETE /identity/v3/users/9d7e9a2846cc4ff3b64a8551a7a733a2 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:38.214565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26a48a3018d44488af3db6ba46e1f38'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.230007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1279/2555] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:37 2022] DELETE /identity/v3/users/c26a48a3018d44488af3db6ba46e1f38 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:38.507499 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:38.508981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:38.548650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:38.584022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8e08d-a08a-42de-b2d2-3cbf09379308 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:38.672366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:38.673822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:38.674105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:38.674383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] PATH_INFO: `/v3/projects/8e73450992ac46049edabd7000d44bde` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:38.675636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e73450992ac46049edabd7000d44bde)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:38.677609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:38.846485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e73450992ac46049edabd7000d44bde', 'request_id': 'req-ad313562-4232-4667-a0e4-b1a01fe6ff58'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.852169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e73450992ac46049edabd7000d44bde', 'request_id': 'req-ad313562-4232-4667-a0e4-b1a01fe6ff58'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:38.876805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:38.879128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:38.894014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c9f09-bc36-4ac6-903c-caac486ba4e3 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:38.960933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:38.979621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:38.979621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:38.979621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] PATH_INFO: `/v3/projects/b2277c37a0674476ba2d1dd1b59125c8` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:38.979621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2277c37a0674476ba2d1dd1b59125c8)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:38.979621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.007503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e73450992ac46049edabd7000d44bde', 'request_id': 'req-ad313562-4232-4667-a0e4-b1a01fe6ff58'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.020763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] The token cache is being invalidate because project 8e73450992ac46049edabd7000d44bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:39.021053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.021717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad313562-4232-4667-a0e4-b1a01fe6ff58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.023201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1277/2556] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:38 2022] DELETE /identity/v3/projects/8e73450992ac46049edabd7000d44bde => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:39.028201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc8e19-d538-4f06-b6a2-b02e410d5a80 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.028484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc8e19-d538-4f06-b6a2-b02e410d5a80 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.028756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc8e19-d538-4f06-b6a2-b02e410d5a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.070931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d8dc8e19-d538-4f06-b6a2-b02e410d5a80 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:39.072370 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d8dc8e19-d538-4f06-b6a2-b02e410d5a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:39.081369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1278/2557] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:39.089928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.090242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.090514 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.109851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:39.111081 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:39.119428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1279/2558] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:39.123723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.135991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1a5c9-10b2-4369-8163-d9c458561d9a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.138644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2277c37a0674476ba2d1dd1b59125c8', 'request_id': 'req-c4f36cbc-5d77-4458-a888-69604d0c9631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.146201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2277c37a0674476ba2d1dd1b59125c8', 'request_id': 'req-c4f36cbc-5d77-4458-a888-69604d0c9631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.152557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2277c37a0674476ba2d1dd1b59125c8', 'request_id': 'req-c4f36cbc-5d77-4458-a888-69604d0c9631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.193301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.193949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.194226 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.194494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.195147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:39.195448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.197122 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.209570 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1280/2559] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:28:39.213974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.223352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8908-8dd4-432a-84ff-4a46c209f5c2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.228180 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.228761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.229037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.229308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.229924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.231899 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.275323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] The token cache is being invalidate because project b2277c37a0674476ba2d1dd1b59125c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:39.275719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.276460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-c4f36cbc-5d77-4458-a888-69604d0c9631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:39.278130 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1280/2560] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:38 2022] DELETE /identity/v3/projects/b2277c37a0674476ba2d1dd1b59125c8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:39.282812 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989779d7-c081-4cb9-984f-3b8d5ac6d392 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.283092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989779d7-c081-4cb9-984f-3b8d5ac6d392 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.283365 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989779d7-c081-4cb9-984f-3b8d5ac6d392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.303642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-989779d7-c081-4cb9-984f-3b8d5ac6d392 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:39.307925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-989779d7-c081-4cb9-984f-3b8d5ac6d392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:39.312879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1281/2561] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:39.321498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.321773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.322045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.341328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:39.342524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:39.350918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1282/2562] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:39.369906 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.379832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d838b4-60f3-45c7-9f39-e76496656661 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.384592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.385183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.385455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.385728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.386366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:39.386662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.388364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.400045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1283/2563] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:28:39.405000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.414997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de7da-0360-4975-b917-fa8d3eb1270b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.419703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.420211 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1281/2564] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:39.430004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.430004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.430004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.430004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.430004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.433296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.435049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:39.498612 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be165-5707-4303-9bf2-1d965c5b634f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.504394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.505005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.505283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.505555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.506186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.508179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.641249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1284/2565] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:39.645247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.703722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1282/2566] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:39.708388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d15d-30ff-467e-9adc-10bcb126a0cc admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.713530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.723925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.723925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.723925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.723925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.723925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.727236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.738598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ec1f-af29-4df4-bcd9-a216805c09f6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.743380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.744027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.744310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.744583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.745216 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.746848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.757224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1283/2567] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:39.761282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.770985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6e7f-0620-405e-b853-8fe0be726b3f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.775817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.776446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.776722 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.776997 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.777893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.779501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.791691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1284/2568] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:39.795903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.806767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8604326-b953-43a6-8a02-ed82b0e81dc8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.811621 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.812350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.812631 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.812913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] PATH_INFO: `/v3/projects/9757c1b18f9d41b4bc6a88cae734cb8e/users/f8a9c50fa8e045689decb316fe9244e0/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.828635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9757c1b18f9d41b4bc6a88cae734cb8e, user_id=f8a9c50fa8e045689decb316fe9244e0, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.831347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3177d1-f227-48e1-9e9f-6329d39b46db admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.851586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1285/2569] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:39.855466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.866332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714b19-a324-4347-88de-7bc4911c2cb7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.871125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.871778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.871778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.872458 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.873060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.874656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.884630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1286/2570] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:39.894226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.904258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56737bd5-7692-4caf-bc1e-347c00a722cb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.910021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.910635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.910914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.911182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.911847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.913473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.925487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1287/2571] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:39.930657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:39.941299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fd24-5fef-46a6-935b-a202a0cbf8bb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:39.946242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:39.946901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.947183 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.947456 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] PATH_INFO: `/v3/projects/21f856c6ca8346c4b942e12a896844d1/users/9c3798e79e1c4e51a19721fd2f940631/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:39.954714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1285/2572] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:39 2022] PUT /identity/v3/projects/9757c1b18f9d41b4bc6a88cae734cb8e/users/f8a9c50fa8e045689decb316fe9244e0/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:39.963462 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f856c6ca8346c4b942e12a896844d1, user_id=9c3798e79e1c4e51a19721fd2f940631, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:39.966170 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef85a9-4b7f-4fe1-a4d9-f04be1dff6f1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:39.973187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:39.973565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:39.973852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.003153 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] MFA Rules not processed for user `f8a9c50fa8e045689decb316fe9244e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:40.004490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:40.067913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1286/2573] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:40.079518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.080930 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:40.103710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1288/2574] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:39 2022] PUT /identity/v3/projects/21f856c6ca8346c4b942e12a896844d1/users/9c3798e79e1c4e51a19721fd2f940631/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:40.115265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.115591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.115901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.147106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] MFA Rules not processed for user `9c3798e79e1c4e51a19721fd2f940631`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:40.148277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:40.191154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f4ba9-ced7-4b2a-ade8-6819c7652f4a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.211166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:40.221021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1289/2575] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:40.225488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.257515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.258172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.260699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.260699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.286725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846692e-533b-43a6-a93c-6ee1e21dd008 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.291638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.292279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.292557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.292828 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.294904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.295380 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.315468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.317104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.326023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1287/2576] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:40.429722 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1290/2577] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:40 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:40.433964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.445149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2621d-3cb0-4fd9-abea-e1df278f8bf2 service nova] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.503676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.504385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.504663 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.504935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.505585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.507551 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.614397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1288/2578] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:40 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:40.618392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.628761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d3d5-644a-472b-8a6e-a97e5268ee65 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.633368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.633966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.634242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.634512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.635149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.636759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.647235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1291/2579] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:40.651247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.661938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21b424-db2b-4132-aef5-17ffdd22b038 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.666934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.667480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.667760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.668182 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.668869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.670463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab45d6-64d2-4cd5-a954-40ccccffd29e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.680856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1289/2580] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:40.684799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:40.694685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d24d-0d78-4317-8b92-a6a453cb061d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:40.699675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:40.700622 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.700764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.700987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] PATH_INFO: `/v3/projects/2a0330bd648e40f1a796cc6873ca1e4c/users/78f9b45e0b4a49ba83833c531c4e5b67/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.716627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0330bd648e40f1a796cc6873ca1e4c, user_id=78f9b45e0b4a49ba83833c531c4e5b67, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:40.719296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:40.909447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1292/2581] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:40 2022] PUT /identity/v3/projects/2a0330bd648e40f1a796cc6873ca1e4c/users/78f9b45e0b4a49ba83833c531c4e5b67/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:40.920526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:40.921004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:40.921004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:40.949350 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] MFA Rules not processed for user `78f9b45e0b4a49ba83833c531c4e5b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:40.950609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:41.025343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1290/2582] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:41.030001 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.093340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd03513-5ac5-48b0-a9bc-42df0e018f53 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.098604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.099203 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.099481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.099749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.100417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.102346 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.263461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1293/2583] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:41 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:41.264735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.275827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad7f8-ffe5-4022-a94f-54408f0a72d6 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.335024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.335816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.336127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.336404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.337065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.339064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.384616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:41.502649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1291/2584] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:41 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:41.507197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.518605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07244b50-42c7-4cfa-80d6-726d0fb11b68 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.523131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.523731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.524053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.524355 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.525061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.526755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.538083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1294/2585] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:41.542293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.552733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c18a-15e1-481b-87c9-115cedb3a91e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.557555 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.558217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.558493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.558768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] PATH_INFO: `/v3/projects/0c4caf1db4ea47528a0513159def145f/users/f2257c822ace46d19ff80a330bd9119c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.573587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4caf1db4ea47528a0513159def145f, user_id=f2257c822ace46d19ff80a330bd9119c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.576298 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.699219 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1292/2586] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:41 2022] PUT /identity/v3/projects/0c4caf1db4ea47528a0513159def145f/users/f2257c822ace46d19ff80a330bd9119c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:41.703190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.713617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171615-ffe4-4aa5-be30-9c4886b10388 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.771091 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.771698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.772008 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.772281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.772913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.774569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e3cb4-8028-4571-86d8-d1f3845fd4df admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.785193 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1295/2587] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:41.789411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.799645 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea5be-dc27-4e5a-9cf1-1c6e949ca433 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:41.804229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:41.804877 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.805692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.805692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] PATH_INFO: `/v3/projects/0c4caf1db4ea47528a0513159def145f/users/f2257c822ace46d19ff80a330bd9119c/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.807294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4caf1db4ea47528a0513159def145f, user_id=f2257c822ace46d19ff80a330bd9119c, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:41.809962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:41.820412 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1293/2588] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:41 2022] PUT /identity/v3/projects/0c4caf1db4ea47528a0513159def145f/users/f2257c822ace46d19ff80a330bd9119c/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:41.831543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:41.831829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:41.832139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:41.860342 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] MFA Rules not processed for user `f2257c822ace46d19ff80a330bd9119c`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:41.861564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:41.927908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1296/2589] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:41.940972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:41.942712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:42.002008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25ab3-3cc4-4bbd-be4f-0565fe4b3c74 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:42.005756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:42.006359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:42.006630 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:42.006896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:42.015951 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:42.018021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:42.027190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1294/2590] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:42.209233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:42.210644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:42.223191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8a12e-282a-4d10-b944-a04112e41e6e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:42.226883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:42.227487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:42.227820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:42.228138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:42.287691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:42.289198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:42.301274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1297/2591] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:42.333726 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:43.536717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:44.050859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:44.113465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:44.212847 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:44.241638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:50.143988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:50.146374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:50.205510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63e8f-cbf9-4143-966b-22dca47009b9 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:50.210201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:50.210828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:50.211074 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6859616-0daf-4a7c-a41b-64acb1bf27bb service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:50.211222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:50.211365 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] PATH_INFO: `/v3/users/f8a9c50fa8e045689decb316fe9244e0` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:50.212818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a9c50fa8e045689decb316fe9244e0)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:50.214742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:50.216242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:50.217045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:50.217329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:50.217604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] PATH_INFO: `/v3/users/9c3798e79e1c4e51a19721fd2f940631` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:50.218765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3798e79e1c4e51a19721fd2f940631)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:50.221251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:50.518598 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a9c50fa8e045689decb316fe9244e0'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.558305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3798e79e1c4e51a19721fd2f940631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.562055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3798e79e1c4e51a19721fd2f940631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.570789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3798e79e1c4e51a19721fd2f940631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.577877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3798e79e1c4e51a19721fd2f940631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.618431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a9c50fa8e045689decb316fe9244e0'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.619068 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a9c50fa8e045689decb316fe9244e0'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.624823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:50.626839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a9c50fa8e045689decb316fe9244e0'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.639280 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2f3a97c5-ea85-4bfb-8629-821127d98ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a9c50fa8e045689decb316fe9244e0'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.645852 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1295/2592] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:50 2022] DELETE /identity/v3/users/f8a9c50fa8e045689decb316fe9244e0 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:50.650487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:50.650774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:50.651083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:50.671151 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:50.672526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:50.730798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1296/2593] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:50.741235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:50.742615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:50.749723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-1dc51489-3cee-4184-b85e-17134aa900dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3798e79e1c4e51a19721fd2f940631'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:50.756444 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1298/2594] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:50 2022] DELETE /identity/v3/users/9c3798e79e1c4e51a19721fd2f940631 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:50.767034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:50.767359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:50.767701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:50.804600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c532f-d11e-4c3e-a12b-960de16c20ab None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:50.860165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:50.860806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:50.861116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:50.861407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:50.863586 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:50.914654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:50.916318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:50.929199 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:50.933805 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:50.935366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:50.946170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1297/2595] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:50.978018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1299/2596] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:50.987686 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:50.993178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.004310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f78-e07d-497c-ad23-1dd0566ae831 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.008429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.009257 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.009368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.009756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.021317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.027638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:51.033569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1298/2597] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:51.475274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:51.476742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.489432 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4577-7fd7-4ccf-a20b-7bc79f191c5e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.497796 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:51.498323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.498440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.498772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.499049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] PATH_INFO: `/v3/projects/9757c1b18f9d41b4bc6a88cae734cb8e` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.499232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.500243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9757c1b18f9d41b4bc6a88cae734cb8e)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.502244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:51.512063 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:51.526076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f423b-5a62-43d1-8557-edd0ac230711 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.530934 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.532193 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.532466 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.532742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] PATH_INFO: `/v3/projects/21f856c6ca8346c4b942e12a896844d1` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.534477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f856c6ca8346c4b942e12a896844d1)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.538203 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:51.682100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9757c1b18f9d41b4bc6a88cae734cb8e', 'request_id': 'req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.688585 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9757c1b18f9d41b4bc6a88cae734cb8e', 'request_id': 'req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.694421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f856c6ca8346c4b942e12a896844d1', 'request_id': 'req-2cbab70d-936a-489a-967b-960342ea2e53'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.695323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9757c1b18f9d41b4bc6a88cae734cb8e', 'request_id': 'req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.701014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f856c6ca8346c4b942e12a896844d1', 'request_id': 'req-2cbab70d-936a-489a-967b-960342ea2e53'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.802660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] The token cache is being invalidate because project 9757c1b18f9d41b4bc6a88cae734cb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:51.802945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.803605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-8f2bf02e-5567-4067-a0dd-3e37fd7f9699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.805101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1300/2598] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:51 2022] DELETE /identity/v3/projects/9757c1b18f9d41b4bc6a88cae734cb8e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:51.809843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471c0ca-be52-4944-81c7-06d0537d134a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.810115 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471c0ca-be52-4944-81c7-06d0537d134a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.810379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471c0ca-be52-4944-81c7-06d0537d134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.832617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b471c0ca-be52-4944-81c7-06d0537d134a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:51.833793 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b471c0ca-be52-4944-81c7-06d0537d134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.855987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f856c6ca8346c4b942e12a896844d1', 'request_id': 'req-2cbab70d-936a-489a-967b-960342ea2e53'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.869067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] The token cache is being invalidate because project 21f856c6ca8346c4b942e12a896844d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:51.869359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.870038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:51.871607 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1299/2599] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:51 2022] DELETE /identity/v3/projects/21f856c6ca8346c4b942e12a896844d1 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:51.876091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:51.877654 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.897147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1301/2600] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:51.904976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.905259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.905538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.927801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:51.929100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:51.937113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1302/2601] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:51.941246 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:51.946047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbab70d-936a-489a-967b-960342ea2e53 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.950628 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a766e-3590-49c8-a305-1a060236dd99 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.950886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.951521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.951800 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.952100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] PATH_INFO: `/v3/users/78f9b45e0b4a49ba83833c531c4e5b67` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.953755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f9b45e0b4a49ba83833c531c4e5b67)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.955878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:51.956003 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.956475 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.956753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.957026 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.957668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:51.957966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.959529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:51.971966 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1303/2602] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:28:51.976027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:51.986231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e91e-4dac-4753-ae2e-3f0c1733e439 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:51.991222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:51.992905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:51.992905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:51.992905 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:51.993776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:51.994965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.105632 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1304/2603] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:51 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:52.109860 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.136853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f9b45e0b4a49ba83833c531c4e5b67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:52.171425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:52.176362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f460-1ea6-4143-908e-c5b1ba087e69 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.181255 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.181866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.182146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.182420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.183056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.185080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.207018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f9b45e0b4a49ba83833c531c4e5b67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:52.207673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f9b45e0b4a49ba83833c531c4e5b67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:52.210338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:52.227933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f9b45e0b4a49ba83833c531c4e5b67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:52.228521 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f9b45e0b4a49ba83833c531c4e5b67'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:52.234397 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1300/2604] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:51 2022] DELETE /identity/v3/users/78f9b45e0b4a49ba83833c531c4e5b67 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:52.355923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1305/2605] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:52 2022] POST /identity/v3/users => generated 303 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:52.360356 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.361782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:52.427839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc34c-e465-429b-a768-084bf4154319 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.432857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.433493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.433799 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.434072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.434735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.436443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.447451 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1301/2606] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:52.451525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.462919 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a6340-4604-444e-bd3d-4c24c484479b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.467831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.468467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.468748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.469045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.469717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.471315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8d859-2b54-4f41-baf3-3b35c189cca5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.482584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1306/2607] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:52.487039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.498250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e79a9-1b89-4fa6-b479-bf34df5cd2f3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.504781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.504781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.504781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.505368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] PATH_INFO: `/v3/projects/932b89ce91ea4239a229843ba398927c/users/dd01879062ef4418909bbc5b6645db84/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.522595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=932b89ce91ea4239a229843ba398927c, user_id=dd01879062ef4418909bbc5b6645db84, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.525328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.697135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1302/2608] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:52 2022] PUT /identity/v3/projects/932b89ce91ea4239a229843ba398927c/users/dd01879062ef4418909bbc5b6645db84/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:52.709637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.709839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.710107 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.739534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] MFA Rules not processed for user `dd01879062ef4418909bbc5b6645db84`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:52.740878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:52.819734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1307/2609] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:52.823755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.879056 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:52.880977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:52.897556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eab90e-b07d-4074-9870-2331c481c52a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.906502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.907405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95e1b-cc5c-4310-b621-e694ecb35bb0 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:52.908897 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.909355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.909810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.911028 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.914575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:52.914801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:52.915774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:52.928517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:52.928517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] PATH_INFO: `/v3/projects/2a0330bd648e40f1a796cc6873ca1e4c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:52.928517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0330bd648e40f1a796cc6873ca1e4c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:52.928517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.037619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1303/2610] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:52 2022] POST /identity/v3/projects => generated 403 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:53.043023 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.054460 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0330bd648e40f1a796cc6873ca1e4c', 'request_id': 'req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.061253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0330bd648e40f1a796cc6873ca1e4c', 'request_id': 'req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.070175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0330bd648e40f1a796cc6873ca1e4c', 'request_id': 'req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.083043 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:53.119390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b48e-4e34-42b0-93ad-89b3243ce0e5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.125071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.131969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.131969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.131969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.131969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.131969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.199727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] The token cache is being invalidate because project 2a0330bd648e40f1a796cc6873ca1e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:53.200226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.200737 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.203467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1308/2611] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:52 2022] DELETE /identity/v3/projects/2a0330bd648e40f1a796cc6873ca1e4c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:53.206556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.208083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:53.228256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1304/2612] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:53 2022] POST /identity/v3/users => generated 308 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:53.232759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.235168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:53.252755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03263b7f-6a94-40c3-96b8-be8a3a9c27f2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.254689 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.255296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.255743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.256049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.256699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.258495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.261761 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:53.271987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1305/2613] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:53.282485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.282641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6ffc4-a235-4ba7-90fd-f3c4649ffc1d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.283749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.283977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e41b8-49d9-4e5c-b2b9-71d6481437b3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.284545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.284886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.285156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] PATH_INFO: `/v3/users/f2257c822ace46d19ff80a330bd9119c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.286271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2257c822ace46d19ff80a330bd9119c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.288199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.288982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.289817 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.290100 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.290819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] PATH_INFO: `/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c/users/99489e8506394acbbde70472ea1e6633/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.309234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdb8ae8e4db42e9864af750dbce8f9c, user_id=99489e8506394acbbde70472ea1e6633, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.312032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.416375 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1306/2614] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:53 2022] PUT /identity/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c/users/99489e8506394acbbde70472ea1e6633/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:53.420462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.484713 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c688-0900-4a2b-a36a-c27c6160b95a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.489640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.490240 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.490516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.490792 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.491441 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.493083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.502710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1307/2615] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:28:53.506780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.516655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db40f1-245b-4407-810b-028c4161b551 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.521334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.521960 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.522237 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.522543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] PATH_INFO: `/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c/users/99489e8506394acbbde70472ea1e6633/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.524503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdb8ae8e4db42e9864af750dbce8f9c, user_id=99489e8506394acbbde70472ea1e6633, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.527129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6aad6-190c-43eb-a979-95bd0aaeebc1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.539177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1308/2616] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:53 2022] PUT /identity/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c/users/99489e8506394acbbde70472ea1e6633/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:53.549575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.549858 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.550136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.579435 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] MFA Rules not processed for user `99489e8506394acbbde70472ea1e6633`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:53.580729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:53.606513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2257c822ace46d19ff80a330bd9119c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.607394 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2257c822ace46d19ff80a330bd9119c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.620689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2257c822ace46d19ff80a330bd9119c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.628049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2257c822ace46d19ff80a330bd9119c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.646793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:53.657616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1309/2617] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:53.668478 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:53.670021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:53.704551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2257c822ace46d19ff80a330bd9119c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:53.710939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1309/2618] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:53 2022] DELETE /identity/v3/users/f2257c822ace46d19ff80a330bd9119c => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:53.729530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc08580-1123-4c08-90a9-7e8b16dcde9c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:53.790477 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:53.791002 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:53.791277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:53.791547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:53.793318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:53.860903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:53.862351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:53.872159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1310/2619] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:53 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:54.031689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.033450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.098943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929e8cf-f2fd-45d3-90c6-93e6fc1ec221 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.103469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.104202 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.104477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.104744 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] PATH_INFO: `/v3/users/dd01879062ef4418909bbc5b6645db84` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.105861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd01879062ef4418909bbc5b6645db84)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.107751 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:54.257374 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.259325 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.272506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227f23-4a2a-4fbf-9ba2-23db5d41ddeb service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.277797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.278611 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.278905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.279172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] PATH_INFO: `/v3/projects/0c4caf1db4ea47528a0513159def145f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.280309 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4caf1db4ea47528a0513159def145f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.282261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:54.376378 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd01879062ef4418909bbc5b6645db84'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.376971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd01879062ef4418909bbc5b6645db84'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.388967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd01879062ef4418909bbc5b6645db84'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.390930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:54.396078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd01879062ef4418909bbc5b6645db84'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.416482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4caf1db4ea47528a0513159def145f', 'request_id': 'req-fa090534-d087-4191-b781-e22155e1b1bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.420345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:54.423177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4caf1db4ea47528a0513159def145f', 'request_id': 'req-fa090534-d087-4191-b781-e22155e1b1bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.538106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d73c36c3-65c0-4669-a70f-acef015fc2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd01879062ef4418909bbc5b6645db84'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.544691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1310/2620] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:54 2022] DELETE /identity/v3/users/dd01879062ef4418909bbc5b6645db84 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:54.549630 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.549911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.550185 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.574046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:54.575262 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.618189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4caf1db4ea47528a0513159def145f', 'request_id': 'req-fa090534-d087-4191-b781-e22155e1b1bd'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.631378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] The token cache is being invalidate because project 0c4caf1db4ea47528a0513159def145f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:54.631648 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.632316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-fa090534-d087-4191-b781-e22155e1b1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:54.634003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1311/2621] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:54 2022] DELETE /identity/v3/projects/0c4caf1db4ea47528a0513159def145f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:54.639116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1311/2622] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:54.646106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd06a00-73fc-445d-bef9-88f9f4f3cc5e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.646106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd06a00-73fc-445d-bef9-88f9f4f3cc5e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.646106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd06a00-73fc-445d-bef9-88f9f4f3cc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.647132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.648595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.667225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-ebd06a00-73fc-445d-bef9-88f9f4f3cc5e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:54.668730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ebd06a00-73fc-445d-bef9-88f9f4f3cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.676742 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1312/2623] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:54.685242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.685547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.685833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.705939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898797f-45e2-4604-8678-3e8c38ed361d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.706231 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:54.708276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.709840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.710452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.710745 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.711025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.712971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:54.723076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1313/2624] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:54.725847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.725984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.727285 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:54.735144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2800a-9a8d-43aa-ba3e-8477785ae18a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.737108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1312/2625] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:54.741565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.742172 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.742556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.742829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.743485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:54.743789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.745429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:54.758915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1314/2626] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:28:54.763478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.773894 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687884f1-d10e-476c-9c29-cbe79306bf40 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.779054 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.779686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.780032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.780360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.781050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.783364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:54.903266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1313/2627] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:54 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:54.907462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:54.917422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7bc80-b7e7-4b2f-9476-b89fa2b4a7a0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:54.981470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:54.982020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:54.982299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:54.982579 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:54.983218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:54.985735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.093839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1315/2628] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:54 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:55.094987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.106306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d31c1-274f-4c32-ad41-ebd487d9f0aa admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.111399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.112033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.112310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.112665 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.114189 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.115772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.127487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1314/2629] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:55.137743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.148352 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86f45-2370-493d-bd3a-24231d8cc4d7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.153415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.154031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.154311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.154584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.155221 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.156926 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.168044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1316/2630] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:55.172861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.182956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b4bd-8857-4937-b83b-b81f5eae6783 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.187549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.188843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.189119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.189388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] PATH_INFO: `/v3/projects/28af2432b5db474cbb9a315494eb2107/users/cc80abb2922c477b9bd0424066b10420/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.205728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28af2432b5db474cbb9a315494eb2107, user_id=cc80abb2922c477b9bd0424066b10420, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.208551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5388-7c7a-469d-9ef0-f83cb54247f7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.214368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.217116 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:55.233145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc35248-9368-4469-accb-2c5906962bf8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.244545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.244545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.244545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.244545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] PATH_INFO: `/v3/projects/932b89ce91ea4239a229843ba398927c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.245083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932b89ce91ea4239a229843ba398927c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.245083 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.428228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1315/2631] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:55 2022] PUT /identity/v3/projects/28af2432b5db474cbb9a315494eb2107/users/cc80abb2922c477b9bd0424066b10420/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:55.440302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.440582 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.440854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.469516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] MFA Rules not processed for user `cc80abb2922c477b9bd0424066b10420`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:55.471315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:55.490329 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:55.490590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932b89ce91ea4239a229843ba398927c', 'request_id': 'req-01273c2e-330d-438d-8310-c9da70da4905'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:55.496710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932b89ce91ea4239a229843ba398927c', 'request_id': 'req-01273c2e-330d-438d-8310-c9da70da4905'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:55.539077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1316/2632] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:55.543789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.572787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932b89ce91ea4239a229843ba398927c', 'request_id': 'req-01273c2e-330d-438d-8310-c9da70da4905'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:55.586588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] The token cache is being invalidate because project 932b89ce91ea4239a229843ba398927c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:55.586905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:55.587509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:55.589458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1317/2633] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:55 2022] DELETE /identity/v3/projects/932b89ce91ea4239a229843ba398927c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:55.593407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.594781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:55.610469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba7f0-cec8-4cca-a833-d97353301e44 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.615875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:55.662224 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273c2e-330d-438d-8310-c9da70da4905 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.666962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.670247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.670247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.670247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] PATH_INFO: `/v3/users/99489e8506394acbbde70472ea1e6633` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.670247 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99489e8506394acbbde70472ea1e6633)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.671207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.674373 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.675134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.675289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.675549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.676212 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.678084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:55.812328 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1317/2634] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:55 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:55.817230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:55.818872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:55.870922 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:55.883507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99feb1-14d3-4842-8532-bd0d1da1a193 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:55.888277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:55.888920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:55.889200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:55.889469 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:55.890092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:55.892186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.015531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1318/2635] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:55 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:56.016741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.020797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99489e8506394acbbde70472ea1e6633'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.026371 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26d590-6932-46b7-ade5-b66d4b236bb8 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.030953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.032182 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.032457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.032724 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.033382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.034945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.047825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1319/2636] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:56.052253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.062171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541cb19-36cb-4bf0-b100-2a1ff137558b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.067788 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.068488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.068767 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.073768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] PATH_INFO: `/v3/projects/4c7a874c55594f43b081b1f539d94a81/users/ccbf9146726246eaacb2bb8ced150523/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.085044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7a874c55594f43b081b1f539d94a81, user_id=ccbf9146726246eaacb2bb8ced150523, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.087827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.097895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99489e8506394acbbde70472ea1e6633'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.098542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99489e8506394acbbde70472ea1e6633'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.106788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99489e8506394acbbde70472ea1e6633'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.119922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99489e8506394acbbde70472ea1e6633'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.126121 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1318/2637] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:28:55 2022] DELETE /identity/v3/users/99489e8506394acbbde70472ea1e6633 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:56.156191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1320/2638] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:56 2022] PUT /identity/v3/projects/4c7a874c55594f43b081b1f539d94a81/users/ccbf9146726246eaacb2bb8ced150523/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:28:56.160553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.161977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.230593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a19fe-8c1e-4669-a57c-e36e4f05cea0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.273311 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:56.288832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.289559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.289907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.290159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.290797 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.292448 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085c116-2605-4413-a55d-a62768703fb5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.304490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1319/2639] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:56.308817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.319401 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38180d9-3afd-40db-b4dd-a41fde7ccf76 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.324417 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.325082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.325354 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.325625 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] PATH_INFO: `/v3/projects/4c7a874c55594f43b081b1f539d94a81/users/ccbf9146726246eaacb2bb8ced150523/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.327530 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7a874c55594f43b081b1f539d94a81, user_id=ccbf9146726246eaacb2bb8ced150523, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.330391 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.341779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1321/2640] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:56 2022] PUT /identity/v3/projects/4c7a874c55594f43b081b1f539d94a81/users/ccbf9146726246eaacb2bb8ced150523/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:56.412907 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.413184 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.413455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.442281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] MFA Rules not processed for user `ccbf9146726246eaacb2bb8ced150523`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:56.444160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.513949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1320/2641] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:56.526387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.528100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.586269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8910839-c775-4066-808c-a10561f6d6d5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.590257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.590864 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.591139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.591407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.601494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.602929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.612740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1322/2642] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:28:56.632532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.638095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.705723 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e673a-c411-4495-a834-e94f93b6f78c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.710660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.711292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.711566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.711837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] PATH_INFO: `/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.713038 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fdb8ae8e4db42e9864af750dbce8f9c)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:56.714937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:56.832009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fdb8ae8e4db42e9864af750dbce8f9c', 'request_id': 'req-6308be2c-4dba-4e66-ba81-5ae45c0d2714'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.837519 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fdb8ae8e4db42e9864af750dbce8f9c', 'request_id': 'req-6308be2c-4dba-4e66-ba81-5ae45c0d2714'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.871576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:56.881319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996941a0-97b7-4b3e-941a-75e210be6fb3 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:56.927092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fdb8ae8e4db42e9864af750dbce8f9c', 'request_id': 'req-6308be2c-4dba-4e66-ba81-5ae45c0d2714'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.933797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:56.934409 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.934808 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.935078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.937101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.939922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:56.941534 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] The token cache is being invalidate because project 7fdb8ae8e4db42e9864af750dbce8f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:28:56.941914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.942660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-6308be2c-4dba-4e66-ba81-5ae45c0d2714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:28:56.944229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1321/2643] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:28:56 2022] DELETE /identity/v3/projects/7fdb8ae8e4db42e9864af750dbce8f9c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:56.949196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:56.949730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:56.949994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:56.969938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:56.971144 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:56.990302 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:57.001399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.002843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11490847-d5f8-4aa2-ab95-b177543dae6a service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.012455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1323/2644] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:57.039385 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1322/2645] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:57.048666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.048951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.049230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.070918 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:57.072162 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:57.080973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1324/2646] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:57.085339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.097372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471a230-a05a-4325-a8d5-f104fe5d7a2c None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.102336 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.102966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.103251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.103561 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.104284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:28:57.104592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.106255 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.121351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1323/2647] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:28:57.128135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.135579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f5b4-6793-4d7e-9b7c-d3a28cc34044 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.141057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.141725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.142263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.142263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.142909 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.144869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.315309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1325/2648] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:57 2022] POST /identity/v3/projects => generated 363 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:57.320241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.331139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a3651-9aa5-49f3-8338-6fd1864e036c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.389338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.390015 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.390294 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.391202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.391709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.393681 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.537353 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1324/2649] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:57 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:57.539224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.549876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094a68-b8f8-492b-99d3-38a2d6a7e234 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.555537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.556209 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.556491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.556765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.557407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.558996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.570279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1326/2650] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:57.580643 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.590970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb46541-dd4a-4029-bd9d-1ca8ec4bd0dc admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.595590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.596212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.596487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.596754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.597421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.599005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446615-8858-4b3e-85e6-37f64263a5e8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.611510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1325/2651] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:57.615757 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:57.626488 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781951-6e9d-41db-94ca-0fa8bd341d51 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:57.631033 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:57.631677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.631992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.632272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] PATH_INFO: `/v3/projects/ab0304407cd3488485c36c95b5a7252c/users/ff19dd7e7e0847daa1455013dfd41fcb/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.648666 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0304407cd3488485c36c95b5a7252c, user_id=ff19dd7e7e0847daa1455013dfd41fcb, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:57.651508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:57.826877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1327/2652] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:57 2022] PUT /identity/v3/projects/ab0304407cd3488485c36c95b5a7252c/users/ff19dd7e7e0847daa1455013dfd41fcb/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:57.841062 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:57.841567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:57.841813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:57.873230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] MFA Rules not processed for user `ff19dd7e7e0847daa1455013dfd41fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:57.874452 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:57.948592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1326/2653] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:28:57.953192 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.031046 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b047a2-aaa0-4177-a100-06216434d305 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.038423 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.039455 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.040081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.040573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.041539 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.043451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.195824 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1328/2654] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:28:57 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:28:58.199404 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:58.200694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.214788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da05ce5-e3f9-47dc-adfb-70f8b6b261c9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.276233 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.276841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.277130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.277380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.278001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.280012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.416737 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1327/2655] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:28:58 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:28:58.421014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.432602 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666cfa-ba46-4272-98d8-06edd51f3dc1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.437359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.437991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.438268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.438540 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.439179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.440811 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.453052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1329/2656] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:58.457338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.467635 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a771d-1713-4f1d-83cc-799b9fdc5bfa admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.472729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.473388 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.473661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.473955 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] PATH_INFO: `/v3/projects/09c21aff6aac44679874f1cf2962ef02/users/01433f364e8143c9a2733a7e3bf4837f/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.490324 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c21aff6aac44679874f1cf2962ef02, user_id=01433f364e8143c9a2733a7e3bf4837f, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.493264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.642275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1328/2657] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:58 2022] PUT /identity/v3/projects/09c21aff6aac44679874f1cf2962ef02/users/01433f364e8143c9a2733a7e3bf4837f/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:58.646465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.656698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd7b2-1979-44e9-9237-233f37a8932b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.717766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.718294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.718575 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.719468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.720155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.727691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b96c2-dcc9-4fcd-bf81-ccf88e82e0bc admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.727691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:28:58.732866 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1330/2658] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:28:58.737359 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.749443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d678-d293-449d-b714-d7de11dc8b1b admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.754027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.754677 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.754949 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.755220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] PATH_INFO: `/v3/projects/09c21aff6aac44679874f1cf2962ef02/users/01433f364e8143c9a2733a7e3bf4837f/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.757223 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c21aff6aac44679874f1cf2962ef02, user_id=01433f364e8143c9a2733a7e3bf4837f, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:58.759913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:58.771463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1329/2659] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:28:58 2022] PUT /identity/v3/projects/09c21aff6aac44679874f1cf2962ef02/users/01433f364e8143c9a2733a7e3bf4837f/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:28:58.782755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.783041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.783319 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.811505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] MFA Rules not processed for user `01433f364e8143c9a2733a7e3bf4837f`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:28:58.813150 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:58.904921 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1331/2660] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:28:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:28:58.913189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:58.915301 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:28:58.981613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9602d-e012-4cbc-97a9-3913358c9061 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:58.985727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:58.986522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:58.986803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:58.987076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:58.998470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:59.000178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:59.010366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1330/2661] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:28:59.155912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:28:59.166228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb6d7-53ca-42b2-9028-87f5c36ce5f1 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:28:59.170165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:28:59.171220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:28:59.171220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:28:59.171599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:28:59.234227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:28:59.235673 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:28:59.246018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1332/2662] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:28:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:02.019455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:02.020914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:02.073924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:02.079574 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f87-e32b-485f-a6c2-a5b2b70cc80c service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:02.083248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:02.083872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:02.084165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:02.084463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:02.086208 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:02.149300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:02.150787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92402251-a00e-45b8-9066-b38f63971e2e service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:02.161089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1331/2663] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:04.469586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:04.471000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:04.550959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290d60-87a8-4403-8885-bbe6ec7819f8 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:04.556887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:04.557474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:04.558039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:04.558039 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] PATH_INFO: `/v3/users/cc80abb2922c477b9bd0424066b10420` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:04.559650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc80abb2922c477b9bd0424066b10420)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:04.561884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:04.596180 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:04.871593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc80abb2922c477b9bd0424066b10420'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:04.872287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc80abb2922c477b9bd0424066b10420'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:04.886854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc80abb2922c477b9bd0424066b10420'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:04.894832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc80abb2922c477b9bd0424066b10420'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:04.981305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc80abb2922c477b9bd0424066b10420'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:04.989336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1333/2664] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:04 2022] DELETE /identity/v3/users/cc80abb2922c477b9bd0424066b10420 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:04.994650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:04.995183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:04.995183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.016125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:05.017354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.100823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1332/2665] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:05.111768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:05.111768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.183612 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efb276-4956-4665-ba4f-0926cc315fd5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:05.187363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:05.188014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:05.188295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:05.188566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.198413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:05.199879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ec58a-c82f-4d94-bf85-f27eedbb8dee service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:05.210257 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1334/2666] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:29:05.275341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:05.277528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.294108 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3081-7220-43a0-9374-26ab6f021e09 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:05.301701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:05.302518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:05.307191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:05.307191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] PATH_INFO: `/v3/users/ff19dd7e7e0847daa1455013dfd41fcb` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.307191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff19dd7e7e0847daa1455013dfd41fcb)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:05.307191 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:05.518343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff19dd7e7e0847daa1455013dfd41fcb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.596103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff19dd7e7e0847daa1455013dfd41fcb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.596744 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff19dd7e7e0847daa1455013dfd41fcb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.605539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff19dd7e7e0847daa1455013dfd41fcb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.618795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff19dd7e7e0847daa1455013dfd41fcb'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.625307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1333/2667] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:05 2022] DELETE /identity/v3/users/ff19dd7e7e0847daa1455013dfd41fcb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:05.629863 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:05.630349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:05.630449 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.650637 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:05.653942 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.716557 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1335/2668] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:05.728696 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:05.730389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.732705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:05.734377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:05.749900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41979478-2709-4bae-be4b-8f7cf217866d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:05.757890 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:05.758750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:05.759141 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:05.759624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] PATH_INFO: `/v3/projects/28af2432b5db474cbb9a315494eb2107` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.763123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28af2432b5db474cbb9a315494eb2107)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:05.764270 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:05.776679 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:05.828646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c295d-982b-49a3-bace-3f71ea4e5918 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:05.832320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:05.832937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:05.833209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:05.833479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:05.843493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:05.844999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:05.855568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1334/2669] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:05.894507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:05.901549 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28af2432b5db474cbb9a315494eb2107', 'request_id': 'req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.907316 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28af2432b5db474cbb9a315494eb2107', 'request_id': 'req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:05.913985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28af2432b5db474cbb9a315494eb2107', 'request_id': 'req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.033282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] The token cache is being invalidate because project 28af2432b5db474cbb9a315494eb2107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:06.033282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.033740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.035282 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1336/2670] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:05 2022] DELETE /identity/v3/projects/28af2432b5db474cbb9a315494eb2107 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:06.039559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:06.049642 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d3c4-be18-4f6c-8841-e39fd05eece5 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:06.120407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:06.121134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:06.121556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:06.121830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] PATH_INFO: `/v3/users/ccbf9146726246eaacb2bb8ced150523` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:06.123431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbf9146726246eaacb2bb8ced150523)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:06.125991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:06.416712 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbf9146726246eaacb2bb8ced150523'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.553178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbf9146726246eaacb2bb8ced150523'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.554073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbf9146726246eaacb2bb8ced150523'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.566423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbf9146726246eaacb2bb8ced150523'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.580528 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbf9146726246eaacb2bb8ced150523'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.587141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1335/2671] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:06 2022] DELETE /identity/v3/users/ccbf9146726246eaacb2bb8ced150523 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:06.615158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:06.617248 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:06.632166 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163a9f-32c0-490c-aaf2-c7191dae9ba1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:06.740468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:06.740468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:06.740468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:06.740468 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] PATH_INFO: `/v3/projects/ab0304407cd3488485c36c95b5a7252c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:06.740988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0304407cd3488485c36c95b5a7252c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:06.740988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:06.876833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0304407cd3488485c36c95b5a7252c', 'request_id': 'req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.882872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0304407cd3488485c36c95b5a7252c', 'request_id': 'req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.889370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0304407cd3488485c36c95b5a7252c', 'request_id': 'req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.998140 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] The token cache is being invalidate because project ab0304407cd3488485c36c95b5a7252c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:06.998431 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:06.999094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.000648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1337/2672] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:06 2022] DELETE /identity/v3/projects/ab0304407cd3488485c36c95b5a7252c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:07.004750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:07.006271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:07.020743 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f6a4-885a-4f7d-86aa-69d2e92229dd admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:07.056305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:07.083785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:07.084459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.084733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.085006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] PATH_INFO: `/v3/users/01433f364e8143c9a2733a7e3bf4837f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.086135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=01433f364e8143c9a2733a7e3bf4837f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:07.088089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:07.166479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:07.168186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:07.182752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e334d6-c3fc-4b85-9ec4-fac1bac0315b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:07.187561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:07.188252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.188533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.188835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] PATH_INFO: `/v3/projects/4c7a874c55594f43b081b1f539d94a81` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.189994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7a874c55594f43b081b1f539d94a81)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:07.192287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:07.338205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01433f364e8143c9a2733a7e3bf4837f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.340983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7a874c55594f43b081b1f539d94a81', 'request_id': 'req-593a7d2d-8609-45d8-bc82-0c36d773ffb6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.343139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:07.347089 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7a874c55594f43b081b1f539d94a81', 'request_id': 'req-593a7d2d-8609-45d8-bc82-0c36d773ffb6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.356181 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7a874c55594f43b081b1f539d94a81', 'request_id': 'req-593a7d2d-8609-45d8-bc82-0c36d773ffb6'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.430572 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01433f364e8143c9a2733a7e3bf4837f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.431293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01433f364e8143c9a2733a7e3bf4837f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.441658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01433f364e8143c9a2733a7e3bf4837f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.457118 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-4bcdc4b0-aee4-4995-92f2-538ab9204eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01433f364e8143c9a2733a7e3bf4837f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.463660 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1336/2673] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:07 2022] DELETE /identity/v3/users/01433f364e8143c9a2733a7e3bf4837f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:07.503984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] The token cache is being invalidate because project 4c7a874c55594f43b081b1f539d94a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:07.504375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.505088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-593a7d2d-8609-45d8-bc82-0c36d773ffb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:07.507700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1338/2674] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:07 2022] DELETE /identity/v3/projects/4c7a874c55594f43b081b1f539d94a81 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:07.512591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.512868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.513152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.535176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:07.536431 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:07.608752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1337/2675] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:07.617924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.618337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.618708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.643800 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:07.645658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:07.653888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1339/2676] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:07.658523 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:07.668876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ede044-7508-4729-90b5-a04d4001b625 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:07.681041 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:07.681704 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.682009 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.682284 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.682947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:29:07.683250 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:07.685051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:07.699633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1338/2677] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:29:07.704677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:07.719211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7340731-fcb1-463b-a800-cb528e86c2eb None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:07.724991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:07.726058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.727194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.728305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.729067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:07.731251 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:07.905108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1340/2678] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:07 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:07.907833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:07.917830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7799a3-e77d-4aab-9754-0c0e76cbddf0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:07.974785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:07.975396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:07.975673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:07.975978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:07.976818 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:07.978701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.041106 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.043988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.059215 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0371637-c204-45ff-90a2-47a655c4ee12 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.064979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.065667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.066043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.066321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] PATH_INFO: `/v3/projects/09c21aff6aac44679874f1cf2962ef02` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.067686 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c21aff6aac44679874f1cf2962ef02)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.070281 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.127668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1339/2679] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:07 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:08.132098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.142785 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830f78-d794-4840-95a6-caf70fc23b59 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.148288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.148961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.149239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.149515 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.150159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.151781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.155831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:08.165346 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1340/2680] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:08.172052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.182064 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b542d4-3c7b-481a-864a-44d3fe830a71 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.186754 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.187894 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.188158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.188433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.189075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.190710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.201527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1341/2681] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:08.212458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.222415 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ee60e-1bd1-469b-8a47-0d634adeb46d admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.227071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.227788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.228496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.228775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] PATH_INFO: `/v3/projects/52df3bb389e449b48d17b67e4c43cb47/users/0426bfe41c814f6dad08fa9dfbf3b7df/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.244462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52df3bb389e449b48d17b67e4c43cb47, user_id=0426bfe41c814f6dad08fa9dfbf3b7df, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.247162 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a718e46-08ed-49f1-87f6-ac0c1e933874 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.248465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c21aff6aac44679874f1cf2962ef02', 'request_id': 'req-2ca4eb76-752a-49c3-b826-affb723f1851'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:08.254574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c21aff6aac44679874f1cf2962ef02', 'request_id': 'req-2ca4eb76-752a-49c3-b826-affb723f1851'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:08.260822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c21aff6aac44679874f1cf2962ef02', 'request_id': 'req-2ca4eb76-752a-49c3-b826-affb723f1851'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:08.343786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1342/2682] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:08 2022] PUT /identity/v3/projects/52df3bb389e449b48d17b67e4c43cb47/users/0426bfe41c814f6dad08fa9dfbf3b7df/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:08.354575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.354956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.355338 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.371869 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] The token cache is being invalidate because project 09c21aff6aac44679874f1cf2962ef02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:08.372176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:08.372939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-2ca4eb76-752a-49c3-b826-affb723f1851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:08.375122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1341/2683] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:08 2022] DELETE /identity/v3/projects/09c21aff6aac44679874f1cf2962ef02 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:08.380014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47406833-068b-4f83-8ee8-e44141c6b17b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.380299 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47406833-068b-4f83-8ee8-e44141c6b17b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.380576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47406833-068b-4f83-8ee8-e44141c6b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.386228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] MFA Rules not processed for user `0426bfe41c814f6dad08fa9dfbf3b7df`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:08.387459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.401313 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-47406833-068b-4f83-8ee8-e44141c6b17b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:08.402538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-47406833-068b-4f83-8ee8-e44141c6b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.454751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1343/2684] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:08.459245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.461746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.463618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1342/2685] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:08.473108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.473382 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.474085 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.476027 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b4867-3be6-45d4-a486-752339bfce70 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.481670 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.482492 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.482626 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.482831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.483518 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.486112 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.498928 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:08.504057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.510517 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1343/2686] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:08.514663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.524086 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24220209-febd-4e24-8db7-aafbfbf35a9e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.530518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.535847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.548032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1344/2687] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:29:08.552226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.563092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d023a9a-25ab-49b0-ac32-45599b19bf69 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.567845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.568520 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.568787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.569062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.569723 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.571749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.588146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:08.591442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1344/2688] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:29:08.595526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.659126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771a2bf-8073-431b-8825-03dd85241374 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.664261 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.664903 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.665179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.665442 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.666093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.668242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.689014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1345/2689] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:29:08.693695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.693695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.791315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ab00d-cf61-4c37-ad81-289d4bca7b28 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.797050 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.797657 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.797936 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.798207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.800022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:08.807905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1345/2690] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:29:08.808979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.815264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.816957 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.819160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e24a3-70aa-429b-808e-aee7a7282502 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.828801 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1346/2691] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:08.829690 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.880752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.881455 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.881735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.882007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.883194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.884924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.915643 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9225-3ba1-470b-b9f5-972ae1afa12c service placement] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.920912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.922155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.922420 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.922684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.923321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:08.925043 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:08.936113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1347/2692] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:29:08.940102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:08.951979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870667-7d36-4ca9-8706-5064ef95c8bb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:08.960015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:08.961157 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:08.961635 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:08.962292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] PATH_INFO: `/v3/projects/3e66562b4ed14f2c95c09508abb11afd/users/67221f60f51b4b098241d5ed25cc66a5/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:08.997831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e66562b4ed14f2c95c09508abb11afd, user_id=67221f60f51b4b098241d5ed25cc66a5, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.002554 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.002987 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:09.011979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1346/2693] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:08 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:09.016751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.027069 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70518677-a40a-4206-83b2-0949ac7fcfa3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.032362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.032978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.033283 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.033694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.034484 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.036160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.051195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1347/2694] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:09.053952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.064495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2731d-1385-415e-b531-123242c70495 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.069165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.069768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.070049 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.070319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.070966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.072911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.083520 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1348/2695] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:09.087954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.110839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1348/2696] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:08 2022] PUT /identity/v3/projects/3e66562b4ed14f2c95c09508abb11afd/users/67221f60f51b4b098241d5ed25cc66a5/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:09.112286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710147-e84d-4b3b-a371-a277209dbd54 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.115656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.190607 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24b396-1559-4225-add6-36aa1add3efb admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.190965 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.191668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.191952 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.192426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] PATH_INFO: `/v3/projects/15a57b6fe0a2485d9ee6e6bc31e243d3/users/b984af290a72425394c560ecb8d349ae/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.195685 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.196842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.196842 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.197067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.197516 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.199116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.210362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a57b6fe0a2485d9ee6e6bc31e243d3, user_id=b984af290a72425394c560ecb8d349ae, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.210749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1349/2697] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:09.215271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.216580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f82a2-3b59-4eb0-977f-52b91c4104bc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.216978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:09.227492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa70e9-0225-425c-9877-738e2b817ec1 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.232230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.232892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.233171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.233445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] PATH_INFO: `/v3/projects/3e66562b4ed14f2c95c09508abb11afd/users/67221f60f51b4b098241d5ed25cc66a5/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.235416 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e66562b4ed14f2c95c09508abb11afd, user_id=67221f60f51b4b098241d5ed25cc66a5, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.238168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efc3a9-79da-4aea-bd83-139129f50173 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.249412 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1350/2698] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:09 2022] PUT /identity/v3/projects/3e66562b4ed14f2c95c09508abb11afd/users/67221f60f51b4b098241d5ed25cc66a5/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:09.260926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.261205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.261477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.291422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] MFA Rules not processed for user `67221f60f51b4b098241d5ed25cc66a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:09.292689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:09.350984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1349/2699] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:09 2022] PUT /identity/v3/projects/15a57b6fe0a2485d9ee6e6bc31e243d3/users/b984af290a72425394c560ecb8d349ae/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:09.361407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.361698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.361976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.363590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1351/2700] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:09.376494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.379714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:09.398500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] MFA Rules not processed for user `b984af290a72425394c560ecb8d349ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:09.399775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:09.422503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:09.440126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9b7b-99ee-4a97-8aef-297c4494e7d2 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.474583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1350/2701] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:09.475502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.498485 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.499266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.499668 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.500070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.512915 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:09.548449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b4381-c4ef-4806-a123-7c839b6ecd9f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.553192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.553802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.554077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.554347 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.554972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.557264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.565163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.566604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.579979 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1352/2702] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:29:09.743848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1351/2703] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:09 2022] POST /identity/v3/projects => generated 381 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:09.745210 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.759234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a9872-41a6-4774-8328-aaac0c91441a service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.768133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.825286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.825898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.826171 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.826439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.827061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.827163 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eeeb-f928-4a84-80eb-212a2ed78f24 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:09.830825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.837878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:09.838507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:09.839017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:09.839158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:09.897688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:09.899143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:09.908851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1352/2704] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:09.978912 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1353/2705] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:09 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:09.983102 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:09.989293 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:10.007751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8b7ae-4c98-4d73-9203-82c2961c6641 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.012973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.013667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.014046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.014319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.014994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.017339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.028587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1353/2706] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:29:10.032974 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.044124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a73dc-f2e2-4bed-bbe9-f76a63cfb4d0 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.048697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.049367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.056795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.056795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] PATH_INFO: `/v3/projects/c267bebe3996458993e81aa3573cc71c/users/17275971c7dd429d8ab6324962968270/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.072515 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c267bebe3996458993e81aa3573cc71c, user_id=17275971c7dd429d8ab6324962968270, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.075777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.227534 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1354/2707] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:10 2022] PUT /identity/v3/projects/c267bebe3996458993e81aa3573cc71c/users/17275971c7dd429d8ab6324962968270/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:10.235134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.246617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d220ef-bc5a-4897-9af5-94b847d887d3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.323334 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.324110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.324235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.324595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.325128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.326863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3960f-31c1-4e76-a146-40c65007c89d admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.337341 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1354/2708] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:29:10.350079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.361459 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed32a5-8bb2-469c-b896-4bb344e3a2ed admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.366524 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.367211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.367470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.367772 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] PATH_INFO: `/v3/projects/c267bebe3996458993e81aa3573cc71c/users/17275971c7dd429d8ab6324962968270/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.369778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c267bebe3996458993e81aa3573cc71c, user_id=17275971c7dd429d8ab6324962968270, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.372482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.384755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1355/2709] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:10 2022] PUT /identity/v3/projects/c267bebe3996458993e81aa3573cc71c/users/17275971c7dd429d8ab6324962968270/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:10.397217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.397606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.397886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.429925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] MFA Rules not processed for user `17275971c7dd429d8ab6324962968270`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:10.431166 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:10.459418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.461465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:10.514687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1355/2710] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:10.515646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.529110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d9f1-d947-4255-8d14-e37430594d5f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.532933 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.533543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.534121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.534121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.535920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:10.550611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:10.580032 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763dc14-4d64-4d46-b67b-2f08521367a4 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.585105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.586057 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.586438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.586786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] PATH_INFO: `/v3/users/0426bfe41c814f6dad08fa9dfbf3b7df` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.588093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0426bfe41c814f6dad08fa9dfbf3b7df)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.590107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.601284 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.602725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.604998 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:10.613556 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1356/2711] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:10.615323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:10.676133 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925f64b-d6b7-4513-bfb9-94a215fb0e93 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:10.680000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:10.680750 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.681097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.681372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.691528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:10.693027 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521d016-91f4-4653-ad84-8b948e098afb service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:10.703648 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1357/2712] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:10.820160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0426bfe41c814f6dad08fa9dfbf3b7df'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:10.892527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0426bfe41c814f6dad08fa9dfbf3b7df'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:10.894344 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0426bfe41c814f6dad08fa9dfbf3b7df'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:10.903000 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0426bfe41c814f6dad08fa9dfbf3b7df'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:10.933440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0426bfe41c814f6dad08fa9dfbf3b7df'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:10.940775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1356/2713] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:10 2022] DELETE /identity/v3/users/0426bfe41c814f6dad08fa9dfbf3b7df => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:10.945216 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:10.945755 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:10.945987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:10.969082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:10.975900 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.053980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1358/2714] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:11.065846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:11.068462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.149553 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5734-e9dd-425f-aa76-b8fabe2a18c0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:11.154048 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:11.154725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:11.155021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:11.155275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:11.167618 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:11.170913 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:11.183881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1357/2715] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:11.357310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:11.358851 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.371032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2614890-c192-48d0-adb0-1f82d6ab355c None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:11.375756 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:11.376661 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:11.376837 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:11.376968 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] PATH_INFO: `/v3/users/b984af290a72425394c560ecb8d349ae` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:11.378194 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b984af290a72425394c560ecb8d349ae)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:11.380264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:11.615058 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b984af290a72425394c560ecb8d349ae'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.615746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b984af290a72425394c560ecb8d349ae'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.617777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:11.630077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b984af290a72425394c560ecb8d349ae'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.637048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b984af290a72425394c560ecb8d349ae'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.697793 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:11.699266 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.714616 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169fe7d-516f-4658-8b52-325b522d6cc9 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:11.719408 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:11.720613 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:11.721073 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:11.721358 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] PATH_INFO: `/v3/projects/52df3bb389e449b48d17b67e4c43cb47` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:11.722659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] RBAC: Authorizing `identity:delete_project(project_id=52df3bb389e449b48d17b67e4c43cb47)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:11.724556 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:11.747949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4373aaed-27af-4739-9acb-2bb6d5e836df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b984af290a72425394c560ecb8d349ae'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.755125 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1359/2716] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:11 2022] DELETE /identity/v3/users/b984af290a72425394c560ecb8d349ae => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:11.759624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:11.759925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:11.760227 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:11.780639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:11.781871 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.859199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1360/2717] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:11.873287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:11.873287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:11.918705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52df3bb389e449b48d17b67e4c43cb47', 'request_id': 'req-df464f12-7dad-491f-a0f2-19782b4f397f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.926600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52df3bb389e449b48d17b67e4c43cb47', 'request_id': 'req-df464f12-7dad-491f-a0f2-19782b4f397f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:11.945399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d865c-159a-481a-a6ad-1494d1451165 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:12.005888 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:12.006597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:12.006885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:12.007158 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:12.029372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52df3bb389e449b48d17b67e4c43cb47', 'request_id': 'req-df464f12-7dad-491f-a0f2-19782b4f397f'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.046366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] The token cache is being invalidate because project 52df3bb389e449b48d17b67e4c43cb47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:12.046789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.048195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.048725 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:12.052990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1358/2718] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:11 2022] DELETE /identity/v3/projects/52df3bb389e449b48d17b67e4c43cb47 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:12.054485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:12.057585 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:12.076061 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:12.078287 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:12.084329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:12.092165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1361/2719] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:12.140726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464f12-7dad-491f-a0f2-19782b4f397f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:12.147542 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:12.148513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:12.148513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:12.150044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] PATH_INFO: `/v3/users/67221f60f51b4b098241d5ed25cc66a5` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:12.150044 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67221f60f51b4b098241d5ed25cc66a5)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:12.152862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:12.433039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67221f60f51b4b098241d5ed25cc66a5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.618891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67221f60f51b4b098241d5ed25cc66a5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.619798 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67221f60f51b4b098241d5ed25cc66a5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.629938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67221f60f51b4b098241d5ed25cc66a5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.643137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67221f60f51b4b098241d5ed25cc66a5'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.649978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1359/2720] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:12 2022] DELETE /identity/v3/users/67221f60f51b4b098241d5ed25cc66a5 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:12.661537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:12.661537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:12.674103 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6cd25-d9c5-43b5-b39e-25c1e88fdcf7 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:12.752589 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:12.753403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:12.753689 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:12.754020 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] PATH_INFO: `/v3/projects/15a57b6fe0a2485d9ee6e6bc31e243d3` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:12.755826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a57b6fe0a2485d9ee6e6bc31e243d3)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:12.758191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:12.912845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a57b6fe0a2485d9ee6e6bc31e243d3', 'request_id': 'req-77477619-e9df-4b96-9f3f-efe982491283'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.923881 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a57b6fe0a2485d9ee6e6bc31e243d3', 'request_id': 'req-77477619-e9df-4b96-9f3f-efe982491283'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:12.934132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a57b6fe0a2485d9ee6e6bc31e243d3', 'request_id': 'req-77477619-e9df-4b96-9f3f-efe982491283'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.019249 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] The token cache is being invalidate because project 15a57b6fe0a2485d9ee6e6bc31e243d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:13.019984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.020538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.023692 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1362/2721] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:12 2022] DELETE /identity/v3/projects/15a57b6fe0a2485d9ee6e6bc31e243d3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:13.027811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:13.029432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:13.047326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9c0e-375f-4a96-9504-bebfbf66fba1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:13.112622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:13.113835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.114059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.114395 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] PATH_INFO: `/v3/users/17275971c7dd429d8ab6324962968270` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.115557 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17275971c7dd429d8ab6324962968270)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:13.117507 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:13.294174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:13.296619 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:13.309578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77477619-e9df-4b96-9f3f-efe982491283 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:13.314766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:13.315405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.315681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.315989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] PATH_INFO: `/v3/projects/3e66562b4ed14f2c95c09508abb11afd` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.317112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e66562b4ed14f2c95c09508abb11afd)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:13.319048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:13.368087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:13.377841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17275971c7dd429d8ab6324962968270'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.469351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17275971c7dd429d8ab6324962968270'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.470342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17275971c7dd429d8ab6324962968270'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.480379 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17275971c7dd429d8ab6324962968270'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.492286 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e66562b4ed14f2c95c09508abb11afd', 'request_id': 'req-d493d341-7c49-4ca3-90be-377d78ec05d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.498104 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6b467bd-0c05-47dd-9567-60e5dc36dc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17275971c7dd429d8ab6324962968270'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.500352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e66562b4ed14f2c95c09508abb11afd', 'request_id': 'req-d493d341-7c49-4ca3-90be-377d78ec05d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.501621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:13.507898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1360/2722] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:13 2022] DELETE /identity/v3/users/17275971c7dd429d8ab6324962968270 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:13.509169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e66562b4ed14f2c95c09508abb11afd', 'request_id': 'req-d493d341-7c49-4ca3-90be-377d78ec05d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.634892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] The token cache is being invalidate because project 3e66562b4ed14f2c95c09508abb11afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:13.635177 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.639727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d493d341-7c49-4ca3-90be-377d78ec05d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:13.639727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1363/2723] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:13 2022] DELETE /identity/v3/projects/3e66562b4ed14f2c95c09508abb11afd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:13.642705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.642988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.643258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.665543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:13.666789 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:13.747424 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1361/2724] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:13.756388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.756766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.757219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.782110 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:13.783543 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:13.798721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1364/2725] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:13.800165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:13.811154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64a4f3-15fe-4415-b26f-4b69f908f723 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:13.817137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:13.817980 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.818433 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.818780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.819449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:29:13.820088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:13.824685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:13.845014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1362/2726] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:29:13.850571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:13.863721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2910ec7-25a1-42bf-bedd-b36ce31f0595 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:13.870739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:13.871845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:13.872433 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:13.872884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:13.873526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:13.876252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.006116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1365/2727] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:13 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:14.014127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.020850 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c0b7-0b40-4070-bf1a-5df7946cd4b3 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.085432 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.086267 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.087107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.087107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.087774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.090020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.135197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.139430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.155535 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1dfde-69d8-4e57-b044-7155e4da7c52 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.160600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.161333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.161739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.162082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] PATH_INFO: `/v3/projects/c267bebe3996458993e81aa3573cc71c` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.163383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c267bebe3996458993e81aa3573cc71c)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.166108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.220600 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1363/2728] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:14.224239 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.236139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62750e41-780d-44cd-a7f3-00788dce7219 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.242819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.243446 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.243726 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.244014 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.244869 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.253019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.265513 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1364/2729] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:14.269423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.283840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271c34f-0f07-46cc-8573-2ce1653c34d8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.290127 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.290853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.291168 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.291526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.292306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.293981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.306588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1365/2730] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:14.310685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.320104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c267bebe3996458993e81aa3573cc71c', 'request_id': 'req-d8bde5e4-3174-40c0-8309-eae81e18662a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:14.320908 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd11189-bc00-4067-afe5-acd926c9e35c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.328447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c267bebe3996458993e81aa3573cc71c', 'request_id': 'req-d8bde5e4-3174-40c0-8309-eae81e18662a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:14.337360 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c267bebe3996458993e81aa3573cc71c', 'request_id': 'req-d8bde5e4-3174-40c0-8309-eae81e18662a'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:14.391831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.392536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.392811 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.393086 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] PATH_INFO: `/v3/projects/68ff4b09339547a4adf08392667b6b7b/users/300247f7531f46b58d23027166dbb143/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.409688 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ff4b09339547a4adf08392667b6b7b, user_id=300247f7531f46b58d23027166dbb143, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.412401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa3ea5-98e6-4948-ba6e-95b52ebaed83 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.423377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] The token cache is being invalidate because project c267bebe3996458993e81aa3573cc71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:14.423712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:14.424364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-d8bde5e4-3174-40c0-8309-eae81e18662a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:14.426570 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1366/2731] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:14 2022] DELETE /identity/v3/projects/c267bebe3996458993e81aa3573cc71c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:14.433113 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21db3-0976-49a8-ac0e-860474c86c3c None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.433392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21db3-0976-49a8-ac0e-860474c86c3c None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.433717 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21db3-0976-49a8-ac0e-860474c86c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.459945 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-3bd21db3-0976-49a8-ac0e-860474c86c3c None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:14.461152 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3bd21db3-0976-49a8-ac0e-860474c86c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.470426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1367/2732] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:14.478835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.479306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.479595 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.482851 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1366/2733] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:14 2022] PUT /identity/v3/projects/68ff4b09339547a4adf08392667b6b7b/users/300247f7531f46b58d23027166dbb143/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:14.498494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.498775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.499052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.501536 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:14.502878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.528327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] MFA Rules not processed for user `300247f7531f46b58d23027166dbb143`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:14.529532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.567994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1368/2734] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:14.574127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.587252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f6ab-1fac-4b00-ad6e-10f1a42e78fd None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.612749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.613861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.614388 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.614733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.615405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:29:14.616336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.618244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.622047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1367/2735] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:14.623079 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.624609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.636353 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1369/2736] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:29:14.637727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.641538 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31714-ad17-4122-8666-49cf3a5438f2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.646905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.647633 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.648289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aff50-624e-4d0b-a856-005ed1927e4a admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.648615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.648615 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.649026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.650986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.653239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.653846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.654124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.654392 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.655012 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.657526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.755603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1368/2737] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:14.759804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.786295 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1370/2738] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:14.787387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.789551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.843634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedda2b-da9e-437b-b15f-0b7626517dd5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.870035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d9b3-7070-495a-8db0-d1b007fdff5b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.875678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.876404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.876753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.877096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.878991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:14.894018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.896131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.908343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1371/2739] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:14.909253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:14.911035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.920705 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.920859 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.920959 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3c2d9-50fd-4569-88a3-e1cef599e27f service placement] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:14.921135 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.922882 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.924878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:14.926035 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:14.928015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:14.929559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:14.929938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:14.930927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:14.933532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.062687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1369/2740] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:15.067071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.074040 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:15.076168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1372/2741] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:14 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:15.079569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.082010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fd15-e652-422d-8be1-a053f29517fc admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.087682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.089235 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.089676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.090149 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.091028 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2b66e-3afc-4016-812c-2d09b4eee1c7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.091251 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.093601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.096604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.097220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.097502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.097779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.098430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.100059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.107174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1370/2742] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:15.111567 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.117351 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1373/2743] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:15.122734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.128939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d78b4-d902-4510-a9e2-0ba1333e2608 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.134213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.134863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.135186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.135476 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.135596 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba8d1-0199-43c3-98a7-0ea7dbdc0799 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.136420 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.138331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.141104 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.141809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.142142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.142484 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.143367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.145082 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.155148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1371/2744] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:15.156263 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.157233 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1374/2745] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:15.167083 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.170059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d9d75-2063-41c1-b279-af2318bfdda4 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.175650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.176500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.176880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.177258 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] PATH_INFO: `/v3/projects/db8c2514e7ad408aacc7c5623f128659/users/7e6d74727cea4ef0a971c87b5cf57cbb/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.178579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5fa8d-0301-484e-a501-11bd8ce0ca24 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.183638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.184359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.184621 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.184964 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] PATH_INFO: `/v3/projects/b781311ee43640d9b485cbc915560b76/users/ec34a575c6c748989e3460350d07e36f/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.196833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8c2514e7ad408aacc7c5623f128659, user_id=7e6d74727cea4ef0a971c87b5cf57cbb, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.199512 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9719fe-a15a-4a9c-a22f-ea0c43813e95 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.207235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b781311ee43640d9b485cbc915560b76, user_id=ec34a575c6c748989e3460350d07e36f, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.209993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7192aa-941b-48eb-bb87-94c21bc4bcab admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.299306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1372/2746] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:15 2022] PUT /identity/v3/projects/db8c2514e7ad408aacc7c5623f128659/users/7e6d74727cea4ef0a971c87b5cf57cbb/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:15.309579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1375/2747] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:15 2022] PUT /identity/v3/projects/b781311ee43640d9b485cbc915560b76/users/ec34a575c6c748989e3460350d07e36f/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:15.310625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.311213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.311844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.314413 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.314701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.314980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.335145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:15.336406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:15.351140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] MFA Rules not processed for user `7e6d74727cea4ef0a971c87b5cf57cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:15.352622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:15.391041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:15.404379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1376/2748] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:29:15.414647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.417073 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:15.432943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1373/2749] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:15.441114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.452505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bac522-83ed-4a56-9db5-9a62203b430f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.457227 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.457831 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.458106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.458372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.459638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.460936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.487886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacd91e-f50a-47c7-8a7d-2b39f7e8f88a None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.491847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.492483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.492760 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.493031 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.503327 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.504787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.514636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1377/2750] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:29:15.567846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1374/2751] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:29:15 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:15.572137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.582309 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdbac-7f39-4015-b7f0-36bb9a98724d service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.649236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.649927 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.650368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.650709 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.651479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.655119 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.955749 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1378/2752] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:29:15 2022] POST /identity/v3/users => generated 299 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:29:15.959890 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:15.970393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5134a-eb36-4c5b-9f92-215457547a1a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:15.979465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:15.980177 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:15.980485 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:15.980756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:15.981401 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:15.983010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:15.994880 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1375/2753] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:29:16.000030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:16.010902 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668d7-7272-4822-9b96-bd08cb4ed2c5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:16.016425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:16.017096 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:16.017362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:16.017636 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] PATH_INFO: `/v3/projects/ba3bbee341854ab8bf33865b7dd267d6/users/6a880690f1e142fa8426ef0478170b26/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:16.033810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bbee341854ab8bf33865b7dd267d6, user_id=6a880690f1e142fa8426ef0478170b26, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:16.036601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:16.149993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1379/2754] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:15 2022] PUT /identity/v3/projects/ba3bbee341854ab8bf33865b7dd267d6/users/6a880690f1e142fa8426ef0478170b26/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:16.153174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:16.163814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ce604-1a88-4441-a1fd-a8edb3196a70 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:16.252160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:16.252967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:16.253915 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:16.254260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:16.254981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:16.257192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cb6f2-8cfe-446c-937b-875584e08703 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:16.270133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1376/2755] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:29:16.274846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:16.289165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60c03-c032-4a86-a53e-4b6409785799 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:16.297656 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:16.298956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:16.299439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:16.300376 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] PATH_INFO: `/v3/projects/ba3bbee341854ab8bf33865b7dd267d6/users/6a880690f1e142fa8426ef0478170b26/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:16.304070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bbee341854ab8bf33865b7dd267d6, user_id=6a880690f1e142fa8426ef0478170b26, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:16.308239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:16.325368 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1380/2756] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:29:16 2022] PUT /identity/v3/projects/ba3bbee341854ab8bf33865b7dd267d6/users/6a880690f1e142fa8426ef0478170b26/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:16.336059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b62a1-1bc1-43d0-928d-98018392e006 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:16.336562 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b62a1-1bc1-43d0-928d-98018392e006 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:16.336671 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b62a1-1bc1-43d0-928d-98018392e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:16.367888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-819b62a1-1bc1-43d0-928d-98018392e006 None None] MFA Rules not processed for user `6a880690f1e142fa8426ef0478170b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:16.369124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-819b62a1-1bc1-43d0-928d-98018392e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:16.449213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1377/2757] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:16.462306 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:16.464070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:16.524613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c5bf-a149-4746-9473-58fbd7ae1a11 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:16.529498 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:16.533476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:16.533476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:16.533476 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:16.596494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:16.597961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:16.607719 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1381/2758] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:16.721312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:17.012663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:17.390436 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:17.391018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:17.391018 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:17.420985 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] MFA Rules not processed for user `ec34a575c6c748989e3460350d07e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:17.422343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:17.512148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1378/2759] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:17.526618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:17.536674 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf81f03-43c5-4cff-8d31-12e3b81f8fb2 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:17.541676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:17.542292 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:17.542576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:17.542849 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:17.555840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:17.555840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:17.571873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1382/2760] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:17.845242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:18.176263 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:18.911517 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:18.913055 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:18.922034 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:18.924076 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:18.926340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03138f4-5059-4b87-b439-4187f70d9c7b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:18.930071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:18.930674 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:18.931201 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:18.931649 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:18.995988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf341f-77bb-4b51-98e2-4b7275ff19e9 service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:19.000418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:19.001334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:19.001871 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6d536-3760-4747-b024-fb4bcc84583b service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:19.001989 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:19.002239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:19.002513 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] PATH_INFO: `/v3/users/a111fd5f36324d75ac8ad55e762aeaca` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:19.003699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a111fd5f36324d75ac8ad55e762aeaca)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:19.005710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:19.013956 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1379/2761] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:19.090296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:19.215301 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a111fd5f36324d75ac8ad55e762aeaca'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.216014 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a111fd5f36324d75ac8ad55e762aeaca'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.227617 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a111fd5f36324d75ac8ad55e762aeaca'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.234331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a111fd5f36324d75ac8ad55e762aeaca'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.330490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a111fd5f36324d75ac8ad55e762aeaca'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.336472 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1383/2762] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:18 2022] DELETE /identity/v3/users/a111fd5f36324d75ac8ad55e762aeaca => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:29:19.594816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:19.595031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:19.596161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:19.617734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] MFA Rules not processed for user `ec34a575c6c748989e3460350d07e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:19.618936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:19.677628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1380/2763] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:19.690691 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:19.693976 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:19.767708 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:19.787920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6383-4dac-45e1-9e77-7a11f2331f30 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:19.792880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:19.793593 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:19.793874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:19.794149 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] PATH_INFO: `/v3/projects/ce2ee7950a334c8888dae733cb698533` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:19.795311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2ee7950a334c8888dae733cb698533)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:19.797848 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:19.939426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2ee7950a334c8888dae733cb698533', 'request_id': 'req-0743ac0c-78ae-4206-bff0-0af705ac1d0d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.941148 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:19.943010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:19.945188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2ee7950a334c8888dae733cb698533', 'request_id': 'req-0743ac0c-78ae-4206-bff0-0af705ac1d0d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:19.952077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2ee7950a334c8888dae733cb698533', 'request_id': 'req-0743ac0c-78ae-4206-bff0-0af705ac1d0d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.002418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76b4d-9dd7-436e-b7ee-2b4bfcbb660e None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:20.053911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:20.054711 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:20.055082 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:20.055462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:20.057494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:20.070910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:20.072787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:20.076765 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] The token cache is being invalidate because project ce2ee7950a334c8888dae733cb698533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:20.077846 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.078500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.080269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1384/2764] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:19 2022] DELETE /identity/v3/projects/ce2ee7950a334c8888dae733cb698533 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:20.085898 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:20.086051 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1381/2765] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:20.087561 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:20.157571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ac0c-78ae-4206-bff0-0af705ac1d0d admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:20.162387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:20.163633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:20.163941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:20.164214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] PATH_INFO: `/v3/users/a73e5b749ea24be8b1c93d14a680b11d` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:20.165448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73e5b749ea24be8b1c93d14a680b11d)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:20.167425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:20.413701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73e5b749ea24be8b1c93d14a680b11d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.414367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73e5b749ea24be8b1c93d14a680b11d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.427345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73e5b749ea24be8b1c93d14a680b11d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.434207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73e5b749ea24be8b1c93d14a680b11d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.498070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73e5b749ea24be8b1c93d14a680b11d'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:20.505137 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1385/2766] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:20 2022] DELETE /identity/v3/users/a73e5b749ea24be8b1c93d14a680b11d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:20.795430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:20.820209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97208044-28f9-493f-a524-b9187d407309 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:20.839622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=100 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:20.865452 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:20.866950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:20.886813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:20.887582 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:20.887967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:20.892893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] PATH_INFO: `/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:20.892893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535204ccbd4b4ad199b8f926f3bde6ee)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:20.892893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:20.925095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2c78f-71a9-409f-8a8a-9e8c70c00998 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:20.928733 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:20.929336 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:20.929608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:20.929878 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:20.931601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:20.996312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:20.997829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:21.007565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1386/2767] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:29:21.051774 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535204ccbd4b4ad199b8f926f3bde6ee', 'request_id': 'req-b26e2ccc-895f-4c19-8949-bb485d822c74'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:21.057174 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535204ccbd4b4ad199b8f926f3bde6ee', 'request_id': 'req-b26e2ccc-895f-4c19-8949-bb485d822c74'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:21.143139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535204ccbd4b4ad199b8f926f3bde6ee', 'request_id': 'req-b26e2ccc-895f-4c19-8949-bb485d822c74'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:21.155025 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] The token cache is being invalidate because project 535204ccbd4b4ad199b8f926f3bde6ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:21.155878 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:21.156614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-b26e2ccc-895f-4c19-8949-bb485d822c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:21.158230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1382/2768] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:20 2022] DELETE /identity/v3/projects/535204ccbd4b4ad199b8f926f3bde6ee => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:21.789579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:25.819652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:25.820924 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:25.835304 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa46a-a447-44d8-938a-ad6d2b9a902d service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:25.924873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:25.924873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:25.924873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:25.924873 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] PATH_INFO: `/v3/users/300247f7531f46b58d23027166dbb143` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:25.925347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=300247f7531f46b58d23027166dbb143)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:25.925347 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:26.212266 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:INSERT=1,keystone:DELETE=7,keystone:SELECT=92 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:26.216980 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300247f7531f46b58d23027166dbb143'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:26.220077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300247f7531f46b58d23027166dbb143'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:26.246701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300247f7531f46b58d23027166dbb143'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:26.263024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300247f7531f46b58d23027166dbb143'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:26.394564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300247f7531f46b58d23027166dbb143'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:26.404529 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1387/2769] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:25 2022] DELETE /identity/v3/users/300247f7531f46b58d23027166dbb143 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:26.406169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:26.406623 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:26.406738 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:26.426421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:26.427715 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:26.500143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1383/2770] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:26.510226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:26.511807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:26.587037 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b6507-ad4c-4aae-9107-53d05c7763a2 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:26.590874 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:26.591522 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:26.591807 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:26.592164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:26.603245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:26.604798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:26.615332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1388/2771] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:29:27.127288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:27.129004 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:27.158443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b22f4-39d4-4a9d-8905-c3db8210492a None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:27.163333 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:27.164134 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:27.164665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:27.165020 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] PATH_INFO: `/v3/projects/68ff4b09339547a4adf08392667b6b7b` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:27.166242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ff4b09339547a4adf08392667b6b7b)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:27.168275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:27.305318 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:27.307053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ff4b09339547a4adf08392667b6b7b', 'request_id': 'req-cad71834-4aff-41a1-ba7f-87b7a2354949'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.314566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ff4b09339547a4adf08392667b6b7b', 'request_id': 'req-cad71834-4aff-41a1-ba7f-87b7a2354949'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.398937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ff4b09339547a4adf08392667b6b7b', 'request_id': 'req-cad71834-4aff-41a1-ba7f-87b7a2354949'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.418423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] The token cache is being invalidate because project 68ff4b09339547a4adf08392667b6b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:27.419077 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.419755 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.425622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1384/2772] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:27 2022] DELETE /identity/v3/projects/68ff4b09339547a4adf08392667b6b7b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:27.426830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:27.437057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8182-ee7f-4046-b53a-d99470e83a27 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:27.503718 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:27.504386 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:27.504669 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:27.504937 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] PATH_INFO: `/v3/users/7e6d74727cea4ef0a971c87b5cf57cbb` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:27.506146 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6d74727cea4ef0a971c87b5cf57cbb)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:27.508385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:27.825445 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6d74727cea4ef0a971c87b5cf57cbb'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.826348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6d74727cea4ef0a971c87b5cf57cbb'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.841127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6d74727cea4ef0a971c87b5cf57cbb'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.843835 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:27.850547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6d74727cea4ef0a971c87b5cf57cbb'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.966437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6d74727cea4ef0a971c87b5cf57cbb'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:27.976578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1389/2773] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:27 2022] DELETE /identity/v3/users/7e6d74727cea4ef0a971c87b5cf57cbb => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:28.522853 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:28.524457 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:28.539702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71834-4aff-41a1-ba7f-87b7a2354949 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:28.614405 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:28.615059 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:28.615500 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:28.615651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] PATH_INFO: `/v3/projects/db8c2514e7ad408aacc7c5623f128659` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:28.617317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8c2514e7ad408aacc7c5623f128659)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:28.619902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:28.831790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8c2514e7ad408aacc7c5623f128659', 'request_id': 'req-bdaa3081-0821-4fa5-9cd1-43888832ba90'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:28.837454 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8c2514e7ad408aacc7c5623f128659', 'request_id': 'req-bdaa3081-0821-4fa5-9cd1-43888832ba90'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:28.900822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8c2514e7ad408aacc7c5623f128659', 'request_id': 'req-bdaa3081-0821-4fa5-9cd1-43888832ba90'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:28.914841 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] The token cache is being invalidate because project db8c2514e7ad408aacc7c5623f128659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:28.915136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:28.916895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:28.920330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1385/2774] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:28 2022] DELETE /identity/v3/projects/db8c2514e7ad408aacc7c5623f128659 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:28.922022 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:28.923562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:28.937939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c98b19-e217-48f5-a41c-1cdaa97b7906 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:29.005573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:29.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:29.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:29.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] PATH_INFO: `/v3/users/6a880690f1e142fa8426ef0478170b26` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:29.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a880690f1e142fa8426ef0478170b26)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:29.014591 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:29.232230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a880690f1e142fa8426ef0478170b26'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:29.232904 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a880690f1e142fa8426ef0478170b26'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:29.245491 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a880690f1e142fa8426ef0478170b26'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:29.252642 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a880690f1e142fa8426ef0478170b26'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:29.358865 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-dbe444a8-95f9-4288-b111-537fabe3e3c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a880690f1e142fa8426ef0478170b26'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:29.365829 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1390/2775] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:29:28 2022] DELETE /identity/v3/users/6a880690f1e142fa8426ef0478170b26 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:29.796840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:29:29.799030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:29.820047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa3081-0821-4fa5-9cd1-43888832ba90 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:29:29.893976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:29:29.894782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:29.895170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:29.895593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] PATH_INFO: `/v3/projects/ba3bbee341854ab8bf33865b7dd267d6` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:29.897095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3bbee341854ab8bf33865b7dd267d6)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:29:29.902204 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:29:29.910245 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:30.076330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3bbee341854ab8bf33865b7dd267d6', 'request_id': 'req-0701e135-340b-4d11-90f2-b4a5ad849bc7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:30.082751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3bbee341854ab8bf33865b7dd267d6', 'request_id': 'req-0701e135-340b-4d11-90f2-b4a5ad849bc7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:30.164110 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3bbee341854ab8bf33865b7dd267d6', 'request_id': 'req-0701e135-340b-4d11-90f2-b4a5ad849bc7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:30.175846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] The token cache is being invalidate because project ba3bbee341854ab8bf33865b7dd267d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:29:30.176169 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:30.176837 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-0701e135-340b-4d11-90f2-b4a5ad849bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:29:30.178378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1386/2776] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:29:29 2022] DELETE /identity/v3/projects/ba3bbee341854ab8bf33865b7dd267d6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:29:39.361731 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=59,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:40.341376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:29:40.820605 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:29:40.821176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:29:40.821176 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:29:40.841168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] MFA Rules not processed for user `9c0c242c0ad048eb8fd35b4671704056`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:29:40.842473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:29:40.905139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1391/2777] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:29:50.897948 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=25 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:30:39.557076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:30:39.557596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:30:39.557596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:30:39.577984 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] MFA Rules not processed for user `ec34a575c6c748989e3460350d07e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:30:39.587360 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:30:39.634872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1387/2778] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:30:39.674092 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:30:39.674773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:30:39.736559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049c350-b572-40b2-bfbc-88c66640ba23 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:30:39.741066 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa271ae95240497e9be3175e44a2b545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:30:39.741677 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:30:39.741953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:30:39.742225 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:30:39.743997 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:30:39.817883 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:30:39.819252 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505babe0-eb39-4ca1-9ed7-23b13ae9d286 service glance] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:30:39.831405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1392/2779] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:30:40.673195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:30:40.674651 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:30:40.743801 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753d78c-805e-4738-8140-497be8d235c2 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:30:40.746052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:30:40.747248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:30:40.747533 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:30:40.747808 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:30:40.749606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:30:40.764321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:30:40.765920 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:30:40.777658 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1388/2780] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:30:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:30:49.826448 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=54 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:30:50.772012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=51 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:31:03.874239 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:31:03.874741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:31:03.874843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:31:03.894453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] MFA Rules not processed for user `9c0c242c0ad048eb8fd35b4671704056`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:31:03.895659 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:31:03.902533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1393/2781] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:31:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:31:13.888437 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:31:47.049525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:31:47.051091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:31:47.063129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1dff-cffe-46c1-afdd-f713374b4593 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:31:47.066942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:31:47.067543 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:31:47.067835 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:31:47.068125 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:31:47.069816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:31:47.080661 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:31:47.082178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:31:47.090593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1389/2782] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:31:53.057362 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:31:53.059490 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:31:53.139017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7189e-2bcb-4836-9065-43a33258ab99 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:31:53.143501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:31:53.144165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:31:53.144440 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:31:53.144710 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] PATH_INFO: `/v3/users/ec34a575c6c748989e3460350d07e36f` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:31:53.145962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec34a575c6c748989e3460350d07e36f)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:31:53.147992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:31:53.388325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec34a575c6c748989e3460350d07e36f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.388992 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec34a575c6c748989e3460350d07e36f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.406616 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec34a575c6c748989e3460350d07e36f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.408787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec34a575c6c748989e3460350d07e36f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.488773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec34a575c6c748989e3460350d07e36f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.496245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1394/2783] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:31:53 2022] DELETE /identity/v3/users/ec34a575c6c748989e3460350d07e36f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:31:53.766617 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:31:53.778938 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8265ff9-62d7-454e-8e56-8d2794965820 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:31:53.840351 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:31:53.840978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:31:53.841256 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:31:53.841529 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] PATH_INFO: `/v3/projects/b781311ee43640d9b485cbc915560b76` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:31:53.842668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b781311ee43640d9b485cbc915560b76)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:31:53.844610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:31:53.991721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b781311ee43640d9b485cbc915560b76', 'request_id': 'req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:53.998544 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b781311ee43640d9b485cbc915560b76', 'request_id': 'req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:54.071758 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b781311ee43640d9b485cbc915560b76', 'request_id': 'req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:54.084573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] The token cache is being invalidate because project b781311ee43640d9b485cbc915560b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:31:54.084863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:54.085548 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:31:54.087146 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1390/2784] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:31:53 2022] DELETE /identity/v3/projects/b781311ee43640d9b485cbc915560b76 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:32:01.482041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:01.483501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:01.558769 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30c1f-2640-4eb8-b2f6-3442705eca45 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:01.571077 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:01.624062 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:01.624678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:01.624951 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:01.625220 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:01.626998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:01.685629 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:01.687222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:01.697563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1395/2785] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:04.081253 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:11.820206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=48 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:19.072417 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:19.074252 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:19.097010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6942fb3-6cdd-43f0-bb18-a5e653913e0a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:19.105241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:19.105241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:19.105241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:19.105241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:19.105803 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:19.120254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:19.122825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:19.135096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1391/2786] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:26.175109 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:26.176541 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:26.233553 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255e5da-e610-4df4-930d-e8930d8ff7cb service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:26.237221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:26.237825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:26.238100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:26.238370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:26.240398 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:26.301969 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:26.303320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:26.312893 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1396/2787] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:29.130030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=6 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:30.405047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:30.406784 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:30.493362 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc0419-80f4-4477-a8f8-01155652711d service nova] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:30.498345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:30.498958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:30.499234 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:30.499506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:30.501355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:30.513421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:30.514988 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:30.527093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1392/2788] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:30.617333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:30.619200 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:30.636088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e34f-4f8c-48c9-a4d9-ac5308fc547c service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:30.641652 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:30.642296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:30.642603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:30.642895 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] PATH_INFO: `/v3/users/9c0c242c0ad048eb8fd35b4671704056` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:30.644117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0c242c0ad048eb8fd35b4671704056)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:30.646097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:30.959045 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0c242c0ad048eb8fd35b4671704056'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:30.959688 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0c242c0ad048eb8fd35b4671704056'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:30.971988 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0c242c0ad048eb8fd35b4671704056'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:30.980080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0c242c0ad048eb8fd35b4671704056'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.087102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-73bb425a-a467-41fc-9b85-c59f135b9ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0c242c0ad048eb8fd35b4671704056'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.094159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1397/2789] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:32:30 2022] DELETE /identity/v3/users/9c0c242c0ad048eb8fd35b4671704056 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:32:31.477321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:31.493094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3dac-3517-42a7-9796-cf960416b001 service placement] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:31.552783 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:31.557668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:31.557668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:31.557668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] PATH_INFO: `/v3/projects/68d8a00f546547178af95e8645c928e7` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:31.557668 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d8a00f546547178af95e8645c928e7)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:31.558071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:31.721829 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d8a00f546547178af95e8645c928e7', 'request_id': 'req-03bfcf90-be3a-4379-9860-52fc16d9b538'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.728161 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d8a00f546547178af95e8645c928e7', 'request_id': 'req-03bfcf90-be3a-4379-9860-52fc16d9b538'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.871012 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d8a00f546547178af95e8645c928e7', 'request_id': 'req-03bfcf90-be3a-4379-9860-52fc16d9b538'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.882914 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] The token cache is being invalidate because project 68d8a00f546547178af95e8645c928e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:32:31.883202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.883874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-03bfcf90-be3a-4379-9860-52fc16d9b538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:32:31.885371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1393/2790] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:32:31 2022] DELETE /identity/v3/projects/68d8a00f546547178af95e8645c928e7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:32:31.890569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:31.891059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:31.891161 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:31.912844 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:32:31.914741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:31.931827 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:31.973269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1398/2791] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:32:31.982606 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:31.982816 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:31.983092 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.004111 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:32:32.005307 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:32.013428 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1394/2792] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:32:32.017671 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.027651 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97504cdf-42dd-4e63-89fe-9114a5c3e0be None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.032411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.032998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.033271 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.033547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.034405 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:32:32.034877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.036913 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.049139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1399/2793] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:32:32.054827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.064106 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392f3e3-109b-43c2-9d44-7b363c5f2cbc None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.068695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.069312 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.069701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.069855 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.070480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.072421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.083734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:32.220217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1395/2794] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:32:32 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:32:32.224631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.234953 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e6d36-4b92-493d-9f22-503459c511ae admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.296228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.296962 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.297235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.297510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.298211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.300300 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.414768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1400/2795] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:32:32 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:32:32.419152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.430473 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223901b-3ded-4759-a2ef-72c48c040ad9 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.435317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.435958 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.439482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.439482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.439482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.439482 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.449090 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1396/2796] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:32:32.454586 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.465663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e5aa-d370-4907-b100-acf33fd8d6bf admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.470289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.470887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.471165 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.471439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.472175 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.473768 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6fec6-687f-41e4-8470-d5fb6100dcfe admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.485310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1401/2797] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:32:32.494735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.505366 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d8cf-b6ea-4573-b855-b64dd7d953b9 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.510605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.511254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.511535 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.511833 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] PATH_INFO: `/v3/projects/8dc7e15961c846888b0deba5f0953f4d/users/2e44c9a1f78a438791e10a7b5ddfb444/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.526843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc7e15961c846888b0deba5f0953f4d, user_id=2e44c9a1f78a438791e10a7b5ddfb444, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.529539 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.613862 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1397/2798] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:32:32 2022] PUT /identity/v3/projects/8dc7e15961c846888b0deba5f0953f4d/users/2e44c9a1f78a438791e10a7b5ddfb444/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:32:32.631613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.631950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.632792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.661603 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] MFA Rules not processed for user `2e44c9a1f78a438791e10a7b5ddfb444`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:32:32.662834 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:32.727996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1402/2799] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:32:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:32:32.733526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.796178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aca88e-a9e2-4a49-a7b5-9d89f73e3f5e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.800822 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:32.801421 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:32.801698 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:32.801971 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:32.802603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:32.805037 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:32.952578 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1398/2800] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:32:32 2022] POST /identity/v3/projects => generated 373 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:32:32.956991 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:32.967660 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee5c1a-09fc-4288-91b7-9e5a715b3197 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:32.980845 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:33.026759 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.027375 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.027712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.027961 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.028600 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.030531 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.163805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1403/2801] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:32:32 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:32:33.167941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.178418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0542e5-984a-48d0-bab1-c1d35c87868c admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.183687 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.187909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.187909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.187909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.187909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.187909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.198005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1399/2802] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:32:33.202230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.212999 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeabe1a-16ba-4c9a-b98e-858e23740905 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.217809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.218483 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.218766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.219036 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] PATH_INFO: `/v3/projects/ca74f54e5c244830bc3fe039d62a579f/users/f8169de4e7a149118cdb4eb28bfb0878/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.234923 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca74f54e5c244830bc3fe039d62a579f, user_id=f8169de4e7a149118cdb4eb28bfb0878, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.237809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.340742 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1404/2803] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:32:33 2022] PUT /identity/v3/projects/ca74f54e5c244830bc3fe039d62a579f/users/f8169de4e7a149118cdb4eb28bfb0878/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:32:33.345093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.355889 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2452a3-4f66-46fb-a0e3-340890917ef7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.412165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.412759 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.413047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.414021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.414587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.416230 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7c9ff-3558-45e4-b96d-e6dfb2917c1b admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.426053 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1400/2804] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:32:33.430474 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.441911 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849bf24-404a-429b-87c6-7c4f3ae4a3db admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.446805 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.447505 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.447813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.448122 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] PATH_INFO: `/v3/projects/ca74f54e5c244830bc3fe039d62a579f/users/f8169de4e7a149118cdb4eb28bfb0878/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.450105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca74f54e5c244830bc3fe039d62a579f, user_id=f8169de4e7a149118cdb4eb28bfb0878, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.452908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.464101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1405/2805] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:32:33 2022] PUT /identity/v3/projects/ca74f54e5c244830bc3fe039d62a579f/users/f8169de4e7a149118cdb4eb28bfb0878/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:32:33.474610 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.475120 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.475494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.510896 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] MFA Rules not processed for user `f8169de4e7a149118cdb4eb28bfb0878`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:32:33.512205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:33.517604 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:33.585694 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1401/2806] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:32:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:32:33.597236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.598940 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:32:33.659069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeeddb-a491-4976-8be5-0880c48dfe7e admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.662802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.663418 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.663684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.664075 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:33.673315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:33.674727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:33.684069 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1406/2807] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:33.925832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:32:33.935787 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792b312-5451-4226-bbe6-9bd8769ee48b None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:32:33.939481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:32:33.940117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:32:33.940390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:32:33.940656 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:32:34.001590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:32:34.003076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d094a-cde7-4718-9b43-87cade83719a service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:32:34.013593 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1402/2808] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:32:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:32:43.679799 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:32:44.009109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=54 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:30.694453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:30.765048 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4da078-b4ea-4ba0-93aa-15fb6684a4bb service cinder] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:30.769662 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:30.770291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:30.770565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:30.771404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] PATH_INFO: `/v3/users/2e44c9a1f78a438791e10a7b5ddfb444` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:30.772584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e44c9a1f78a438791e10a7b5ddfb444)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:30.774499 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:31.032317 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e44c9a1f78a438791e10a7b5ddfb444'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:31.038244 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e44c9a1f78a438791e10a7b5ddfb444'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:31.057084 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e44c9a1f78a438791e10a7b5ddfb444'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:31.064496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e44c9a1f78a438791e10a7b5ddfb444'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:31.178521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e44c9a1f78a438791e10a7b5ddfb444'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:31.185186 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1407/2809] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:33:30 2022] DELETE /identity/v3/users/2e44c9a1f78a438791e10a7b5ddfb444 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:31.189773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:31.190313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:31.190313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:31.211939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:31.213129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:31.276222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1403/2810] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:31.286947 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:31.288489 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:31.357211 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29293-efcd-4405-9047-0a943337d3da admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:31.360925 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:31.361538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:31.361817 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:31.362090 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:31.372507 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:31.373958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:31.384269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1408/2811] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:33:31.893021 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:31.894560 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:31.907463 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7932-80e9-4fda-9ae4-459237d3e9c9 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:31.912483 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:31.913241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:31.913652 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:31.914001 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] PATH_INFO: `/v3/projects/8dc7e15961c846888b0deba5f0953f4d` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:31.915158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc7e15961c846888b0deba5f0953f4d)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:31.917115 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:32.094425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc7e15961c846888b0deba5f0953f4d', 'request_id': 'req-389730b8-f34f-4986-91d3-e95f02556e3e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.100844 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc7e15961c846888b0deba5f0953f4d', 'request_id': 'req-389730b8-f34f-4986-91d3-e95f02556e3e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.189354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc7e15961c846888b0deba5f0953f4d', 'request_id': 'req-389730b8-f34f-4986-91d3-e95f02556e3e'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.202962 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] The token cache is being invalidate because project 8dc7e15961c846888b0deba5f0953f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:33:32.207665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.207665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.207665 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1404/2812] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:33:31 2022] DELETE /identity/v3/projects/8dc7e15961c846888b0deba5f0953f4d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:32.209601 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:32.219943 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cef6e-a760-4359-9af1-4120a51aed93 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:32.242030 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:32.283164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:32.283790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:32.284095 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:32.284372 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] PATH_INFO: `/v3/users/f8169de4e7a149118cdb4eb28bfb0878` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:32.285547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8169de4e7a149118cdb4eb28bfb0878)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:32.288254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:32.592856 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8169de4e7a149118cdb4eb28bfb0878'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.593538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8169de4e7a149118cdb4eb28bfb0878'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.606579 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8169de4e7a149118cdb4eb28bfb0878'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.615265 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8169de4e7a149118cdb4eb28bfb0878'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.749256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-86759fec-16f5-432f-907c-8cd968e87228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8169de4e7a149118cdb4eb28bfb0878'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:32.755831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1409/2813] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:33:32 2022] DELETE /identity/v3/users/f8169de4e7a149118cdb4eb28bfb0878 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:33.225588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:33.227088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:33.242396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389730b8-f34f-4986-91d3-e95f02556e3e admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:33.313129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:33.313757 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:33.314036 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:33.314308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] PATH_INFO: `/v3/projects/ca74f54e5c244830bc3fe039d62a579f` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:33.315418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca74f54e5c244830bc3fe039d62a579f)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:33.317494 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:33.470942 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:33.486977 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca74f54e5c244830bc3fe039d62a579f', 'request_id': 'req-3dc1fa06-13f8-4940-bec3-29fafeed4b93'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:33.493296 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca74f54e5c244830bc3fe039d62a579f', 'request_id': 'req-3dc1fa06-13f8-4940-bec3-29fafeed4b93'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:33.675093 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca74f54e5c244830bc3fe039d62a579f', 'request_id': 'req-3dc1fa06-13f8-4940-bec3-29fafeed4b93'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:33.687526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] The token cache is being invalidate because project ca74f54e5c244830bc3fe039d62a579f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81891) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:33:33.687947 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:33.688510 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-3dc1fa06-13f8-4940-bec3-29fafeed4b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:33.689986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1405/2814] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:33:33 2022] DELETE /identity/v3/projects/ca74f54e5c244830bc3fe039d62a579f => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:33.694822 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:33.695343 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:33.695453 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:33.715112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:33.716427 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:33.776664 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1410/2815] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:33.784761 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:33.785047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:33.785323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:33.810756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:33.811970 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:33.820229 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1406/2816] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:33.824655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:33.834611 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639160e6-c4b3-48fb-8c08-d15bcb357286 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:33.839533 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:33.840297 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:33.840526 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:33.840795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] PATH_INFO: `/v3/domains` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:33.841439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81890) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:33:33.841738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:33.843315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:33.855478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1411/2817] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:33:33.860186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:33.870390 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1e49-c7e5-4c7a-aabf-f09f9e620d8d None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:33.876602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:33.876602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:33.876602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:33.876602 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:33.877087 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:33.879010 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:33.975039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1407/2818] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:33:33 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:33.979151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:33.989693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8530-dd8c-4741-8bb8-1685e0b3e502 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.018633 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:34.052072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.052604 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.052884 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.053155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.053787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.055795 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.156191 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1412/2819] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:33:33 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:34.160682 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.172487 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20577d-f1ca-4742-9837-a098d9674531 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.177187 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.177809 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.178061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.178320 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.178961 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.180622 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.191423 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1408/2820] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:33:34.195564 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.207188 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13709c30-ae20-4ec6-9af4-bf6a8571d74c admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.212548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.213226 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.213538 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.213813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.214461 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.216080 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fcfce-ab2a-496a-911e-0ca105a6889b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.228231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1413/2821] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:34.232469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.243006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1397-7678-4e41-b751-9fbab47e45eb admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.247771 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.248456 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.248734 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.249006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] PATH_INFO: `/v3/projects/5a7da67b8cdf4029ad19cc533af93bbe/users/efb4a8e0b6424978be678e12833fd197/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.274287 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7da67b8cdf4029ad19cc533af93bbe, user_id=efb4a8e0b6424978be678e12833fd197, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.277165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.378989 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1409/2822] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:34 2022] PUT /identity/v3/projects/5a7da67b8cdf4029ad19cc533af93bbe/users/efb4a8e0b6424978be678e12833fd197/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:34.448142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.448425 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.448697 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.477832 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] MFA Rules not processed for user `efb4a8e0b6424978be678e12833fd197`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:34.479044 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:34.546332 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1414/2823] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:33:34.551003 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.613866 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d5d02-0fc8-4b97-b3c4-0ec3f98438e1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.619387 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.623061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.623061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.623061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.623061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.623501 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.717718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1410/2824] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:33:34 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:34.722025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.734018 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecd051-5370-4bcb-ac21-e69a54426b8d None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.791939 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.792550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.792830 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.793102 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] PATH_INFO: `/v3/users` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.793956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.796134 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.944052 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1415/2825] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:33:34 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:34.948326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.959842 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825a324-e951-4a5f-946a-c9fe749ea012 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:34.964619 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:34.965345 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:34.965627 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:34.965898 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:34.966540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:34.968270 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:34.980085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1411/2826] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:34.984501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:34.995470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c84786-5a72-446d-8b57-ce5357d9bb7f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:35.004712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:35.004712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:35.004712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:35.004712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] PATH_INFO: `/v3/projects/38c92a1baa3f4783b426f30fdd3832bb/users/a6e7890960b54220ab5b4ec29df8e624/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:35.018857 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c92a1baa3f4783b426f30fdd3832bb, user_id=a6e7890960b54220ab5b4ec29df8e624, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:35.022057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:35.149732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1416/2827] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:34 2022] PUT /identity/v3/projects/38c92a1baa3f4783b426f30fdd3832bb/users/a6e7890960b54220ab5b4ec29df8e624/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:35.153777 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:35.164380 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9edee-e66f-4f55-a35f-f606c4e93271 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:35.184856 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:35.226269 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:35.226824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:35.227105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:35.227378 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:35.228088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:35.229756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d1a51-e526-42f9-84d6-fcc4e315ad58 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:35.241225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1412/2828] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:35.245479 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:35.251164 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:35.257234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd46ef-a9ca-4483-be23-ff43a4608553 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:35.262015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:35.262667 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:35.262946 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:35.263221 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] PATH_INFO: `/v3/projects/38c92a1baa3f4783b426f30fdd3832bb/users/a6e7890960b54220ab5b4ec29df8e624/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:35.265381 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c92a1baa3f4783b426f30fdd3832bb, user_id=a6e7890960b54220ab5b4ec29df8e624, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:35.268059 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:35.279487 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1417/2829] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:35 2022] PUT /identity/v3/projects/38c92a1baa3f4783b426f30fdd3832bb/users/a6e7890960b54220ab5b4ec29df8e624/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:33:35.289439 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:35.289710 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:35.289986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:35.323206 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] MFA Rules not processed for user `a6e7890960b54220ab5b4ec29df8e624`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:35.324458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:35.398768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1413/2830] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:35.410390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:35.412112 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:35.471464 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b030e-82cf-4caf-b9cf-131d7887da36 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:35.478785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:35.479325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:35.479597 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:35.479880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:35.490321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:35.496639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441c380-8f2d-43a7-b3fd-4897029d07d8 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:35.501057 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1418/2831] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:33:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:33:36.824143 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:36.886480 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbf6ad-70a4-44a5-83f4-8db9f7a0dc1f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:36.891306 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:36.891968 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:36.892241 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:36.892511 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] PATH_INFO: `/v3/users/efb4a8e0b6424978be678e12833fd197` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:36.893641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb4a8e0b6424978be678e12833fd197)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:36.896225 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:37.112154 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb4a8e0b6424978be678e12833fd197'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:37.113824 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:37.212676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb4a8e0b6424978be678e12833fd197'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:37.213414 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb4a8e0b6424978be678e12833fd197'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:37.221133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb4a8e0b6424978be678e12833fd197'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:37.234268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb4a8e0b6424978be678e12833fd197'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:37.241158 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1414/2832] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:33:36 2022] DELETE /identity/v3/users/efb4a8e0b6424978be678e12833fd197 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:37.246739 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:37.247234 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:37.247345 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:37.266411 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:37.267704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:37.334206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1419/2833] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:37.344088 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:37.345502 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:37.403440 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1aa3-8e72-4764-98af-8bdfaa699149 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:37.407067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:37.408369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:37.408369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:37.408823 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:37.417839 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:37.419271 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:37.428625 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1415/2834] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:33:37.911576 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:37.916825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:37.925678 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4696e-b129-48b0-ba47-8c5c39b534e9 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:37.931078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:37.931746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:37.932213 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:37.932481 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] PATH_INFO: `/v3/projects/5a7da67b8cdf4029ad19cc533af93bbe` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:37.933753 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7da67b8cdf4029ad19cc533af93bbe)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:37.935712 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:38.093675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7da67b8cdf4029ad19cc533af93bbe', 'request_id': 'req-4b567ac0-6ecb-4438-8f98-528d9ead527b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.100274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7da67b8cdf4029ad19cc533af93bbe', 'request_id': 'req-4b567ac0-6ecb-4438-8f98-528d9ead527b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.108021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7da67b8cdf4029ad19cc533af93bbe', 'request_id': 'req-4b567ac0-6ecb-4438-8f98-528d9ead527b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.251105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] The token cache is being invalidate because project 5a7da67b8cdf4029ad19cc533af93bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:33:38.251843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.252053 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.253500 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1420/2835] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:33:37 2022] DELETE /identity/v3/projects/5a7da67b8cdf4029ad19cc533af93bbe => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:38.257791 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:38.267902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657e92-fc6b-4ee5-802e-7497ac0328bd service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:38.326337 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:38.326972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:38.327272 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:38.327575 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] PATH_INFO: `/v3/users/a6e7890960b54220ab5b4ec29df8e624` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:38.328832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e7890960b54220ab5b4ec29df8e624)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:38.331196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:38.581273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e7890960b54220ab5b4ec29df8e624'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.852394 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e7890960b54220ab5b4ec29df8e624'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.853022 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e7890960b54220ab5b4ec29df8e624'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.860959 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e7890960b54220ab5b4ec29df8e624'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.866581 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:38.874583 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-5bfc6df3-62c2-4f34-94a4-9f7d4671d512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e7890960b54220ab5b4ec29df8e624'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:38.881173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1416/2836] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:33:38 2022] DELETE /identity/v3/users/a6e7890960b54220ab5b4ec29df8e624 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:39.374127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:39.375569 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:39.394078 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567ac0-6ecb-4438-8f98-528d9ead527b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:39.402990 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:39.450886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:39.451527 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:39.451809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:39.452118 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] PATH_INFO: `/v3/projects/38c92a1baa3f4783b426f30fdd3832bb` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:39.453247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c92a1baa3f4783b426f30fdd3832bb)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:39.455168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:39.652572 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c92a1baa3f4783b426f30fdd3832bb', 'request_id': 'req-7414b400-2445-4d0f-8417-f39db0c80a78'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:39.658843 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c92a1baa3f4783b426f30fdd3832bb', 'request_id': 'req-7414b400-2445-4d0f-8417-f39db0c80a78'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:39.666454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c92a1baa3f4783b426f30fdd3832bb', 'request_id': 'req-7414b400-2445-4d0f-8417-f39db0c80a78'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:39.793963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] The token cache is being invalidate because project 38c92a1baa3f4783b426f30fdd3832bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:33:39.794352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:39.794993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7414b400-2445-4d0f-8417-f39db0c80a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:33:39.796627 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1421/2837] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:33:39 2022] DELETE /identity/v3/projects/38c92a1baa3f4783b426f30fdd3832bb => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:39.802136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:39.802659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:39.802659 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:39.822672 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:39.823922 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:39.885373 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1417/2838] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:39.893013 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:39.893294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:39.893568 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:39.914325 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:39.915590 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:39.924684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1422/2839] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:39.934609 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:39.944289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18e5c8-e21e-411e-8d79-fc6a0cc7e2ec None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:39.949096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:39.949697 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:39.949974 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:39.950383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:39.951340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:33:39.951340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:39.952936 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:39.966273 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1418/2840] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:33:39.970608 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:39.980721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c509df-3112-482f-9ca2-f6597e562bbf None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:39.985359 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:39.985981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:39.986253 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:39.986523 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:39.987154 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:39.989174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.083277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1423/2841] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:33:39 2022] POST /identity/v3/projects => generated 355 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:40.087566 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.097973 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0a30f-70da-4208-a664-01b330d06dbe admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.160372 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.161592 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.161867 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.162133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.162778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.164773 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.338967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1419/2842] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:33:40 2022] POST /identity/v3/users => generated 278 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:40.343163 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.353525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de044424-c66e-4c7e-99c6-0252cc73c8a9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.358528 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.359129 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.359406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.359681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.360403 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.361998 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.372547 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1424/2843] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:40.376699 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.387750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea94f51-1460-4e8d-926c-49e001f14b62 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.392493 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.393117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.393396 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.393669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.394316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.396573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21df494-5b4d-4e0e-8d4f-cf57f03c2a3e admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.408596 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1420/2844] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:40.412839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.423143 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b536-ffb7-464e-a5fb-88902e339ca7 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.427606 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.428294 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.428574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.428847 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] PATH_INFO: `/v3/projects/f367a5ad4b03457bb72a578988ebf345/users/ecdd4392016647ed9ef2e6513e2bde14/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.444920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f367a5ad4b03457bb72a578988ebf345, user_id=ecdd4392016647ed9ef2e6513e2bde14, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.447618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.544148 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1425/2845] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:40 2022] PUT /identity/v3/projects/f367a5ad4b03457bb72a578988ebf345/users/ecdd4392016647ed9ef2e6513e2bde14/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:40.556183 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.556450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.556721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.588248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] MFA Rules not processed for user `ecdd4392016647ed9ef2e6513e2bde14`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:40.589467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:40.630937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:40.654634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1421/2846] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:40.659238 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.706079 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:40.725004 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c858410-7e59-4630-ac6c-faaf0030385b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.729926 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.730566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.730840 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.731105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.731734 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.733701 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:40.848357 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1426/2847] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:33:40 2022] POST /identity/v3/projects => generated 355 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:40.852481 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:40.864392 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5f27a-5f99-43b1-9a89-077e0433d7dd None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:40.920990 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:40.921599 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:40.921873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:40.922147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:40.922781 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:40.924725 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.029418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1422/2848] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:33:40 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:33:41.033450 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:41.044087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdbc8d-61f9-4db1-a7ca-632a1d806d2f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:41.048704 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:41.049303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.054228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.054228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.054228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:41.054228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.066693 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1427/2849] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:41.068709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:41.079254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f692a3-7624-438a-88e2-751312fe3190 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:41.084398 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:41.085094 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.085371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.085644 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] PATH_INFO: `/v3/projects/191680d4f6c149babc7c17c65ec25f99/users/97f7e8d2f33a44b08a2af5cae32600c4/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.101718 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=191680d4f6c149babc7c17c65ec25f99, user_id=97f7e8d2f33a44b08a2af5cae32600c4, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:41.104427 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.264367 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1423/2850] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:41 2022] PUT /identity/v3/projects/191680d4f6c149babc7c17c65ec25f99/users/97f7e8d2f33a44b08a2af5cae32600c4/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:41.268199 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:41.278324 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c5c3-9f78-4a96-a2fb-120c82860cfd admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:41.337349 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:41.337956 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.338231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.338502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.339142 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:41.340920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75681ab2-1947-4c76-9058-261527067afd admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.351470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1428/2851] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:33:41.355667 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:41.367371 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abe399-8ff8-4918-8c26-d1d6ed745e69 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:41.372089 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:41.372756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.373030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.373300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] PATH_INFO: `/v3/projects/191680d4f6c149babc7c17c65ec25f99/users/97f7e8d2f33a44b08a2af5cae32600c4/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.375232 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=191680d4f6c149babc7c17c65ec25f99, user_id=97f7e8d2f33a44b08a2af5cae32600c4, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:41.377905 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.388948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1424/2852] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:33:41 2022] PUT /identity/v3/projects/191680d4f6c149babc7c17c65ec25f99/users/97f7e8d2f33a44b08a2af5cae32600c4/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:33:41.399366 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.399649 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.399944 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.430914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] MFA Rules not processed for user `97f7e8d2f33a44b08a2af5cae32600c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:33:41.432314 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:41.503120 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1429/2853] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:33:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:33:41.514289 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:41.516202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:33:41.575321 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0d12d-27c4-43b4-9c1e-814158f5f1cd admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:41.579195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:41.579802 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:41.580113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:41.580382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:41.590145 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:41.591595 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:41.601709 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1425/2854] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:33:41.994088 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:33:42.005438 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48420f6a-eb71-44be-b5d1-68dd8c8a5fcc None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:33:42.008949 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:33:42.009544 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:33:42.009818 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:33:42.010087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:33:42.044466 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:42.069610 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:33:42.071071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448dfc8-7310-4a75-b846-019ade6e8ff7 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:33:42.083910 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1430/2855] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:33:51.596945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:33:52.143400 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=11 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:02.681505 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:02.746085 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794cc4f-9ddd-4efd-818c-36a88017e353 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:02.750685 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:02.751310 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:02.751589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:02.751911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] PATH_INFO: `/v3/users/ecdd4392016647ed9ef2e6513e2bde14` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:02.753071 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdd4392016647ed9ef2e6513e2bde14)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:02.755109 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:03.129361 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdd4392016647ed9ef2e6513e2bde14'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:03.240826 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdd4392016647ed9ef2e6513e2bde14'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:03.241638 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdd4392016647ed9ef2e6513e2bde14'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:03.249342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdd4392016647ed9ef2e6513e2bde14'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:03.264496 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdd4392016647ed9ef2e6513e2bde14'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:03.277133 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1426/2856] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:34:02 2022] DELETE /identity/v3/users/ecdd4392016647ed9ef2e6513e2bde14 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:34:03.277279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:03.277279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:03.277279 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:03.296566 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:03.297778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:03.358254 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1431/2857] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:03.368516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:03.369937 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:03.428243 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae3d21-9764-49d9-bbb9-99c2736adab8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:03.431806 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:03.432438 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:03.432717 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:03.432986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:03.441887 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:03.443509 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:03.454070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1427/2858] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:03.939291 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:03.940981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:03.960532 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25aa3-3d20-435f-b79b-5fce4cc48ffa None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:03.966714 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:03.967404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:03.967624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:03.967938 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] PATH_INFO: `/v3/projects/f367a5ad4b03457bb72a578988ebf345` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:03.969070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f367a5ad4b03457bb72a578988ebf345)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:03.974379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:04.096897 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f367a5ad4b03457bb72a578988ebf345', 'request_id': 'req-5780910b-d468-4ddb-ab2d-09ed7036265f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.102151 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f367a5ad4b03457bb72a578988ebf345', 'request_id': 'req-5780910b-d468-4ddb-ab2d-09ed7036265f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.109055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f367a5ad4b03457bb72a578988ebf345', 'request_id': 'req-5780910b-d468-4ddb-ab2d-09ed7036265f'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.200303 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] The token cache is being invalidate because project f367a5ad4b03457bb72a578988ebf345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:34:04.200592 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.201229 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.202892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1432/2859] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:34:03 2022] DELETE /identity/v3/projects/f367a5ad4b03457bb72a578988ebf345 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:34:04.207236 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:04.217242 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5931db-6f86-47d0-a1a3-10f8656ef9a3 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:04.235957 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:04.276872 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:04.277498 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:04.277775 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:04.278205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] PATH_INFO: `/v3/users/97f7e8d2f33a44b08a2af5cae32600c4` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:04.279323 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f7e8d2f33a44b08a2af5cae32600c4)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:04.281319 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:04.512945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f7e8d2f33a44b08a2af5cae32600c4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.588532 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f7e8d2f33a44b08a2af5cae32600c4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.589152 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f7e8d2f33a44b08a2af5cae32600c4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.596530 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f7e8d2f33a44b08a2af5cae32600c4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.608840 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-2ea8e1ee-975d-4170-8dab-205b6b58dacd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f7e8d2f33a44b08a2af5cae32600c4'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:04.614614 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1428/2860] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:34:04 2022] DELETE /identity/v3/users/97f7e8d2f33a44b08a2af5cae32600c4 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:05.122108 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:05.123550 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:05.137184 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780910b-d468-4ddb-ab2d-09ed7036265f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:05.195573 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:05.196256 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.196518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.196787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] PATH_INFO: `/v3/projects/191680d4f6c149babc7c17c65ec25f99` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.197920 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191680d4f6c149babc7c17c65ec25f99)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:05.199823 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:05.322235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:05.339242 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191680d4f6c149babc7c17c65ec25f99', 'request_id': 'req-0e6dfcb5-d166-4ea2-b792-8c709520e639'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:05.344803 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191680d4f6c149babc7c17c65ec25f99', 'request_id': 'req-0e6dfcb5-d166-4ea2-b792-8c709520e639'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:05.351207 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191680d4f6c149babc7c17c65ec25f99', 'request_id': 'req-0e6dfcb5-d166-4ea2-b792-8c709520e639'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:05.456774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] The token cache is being invalidate because project 191680d4f6c149babc7c17c65ec25f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:34:05.457070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:05.457732 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-0e6dfcb5-d166-4ea2-b792-8c709520e639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:05.459308 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1433/2861] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:34:05 2022] DELETE /identity/v3/projects/191680d4f6c149babc7c17c65ec25f99 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:05.464450 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.464967 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.465072 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.484874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:05.486721 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:05.548098 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1429/2862] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:05.557658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.568258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.568258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.578613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:05.579813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:05.587728 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1434/2863] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:05.592103 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:05.602011 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937a703-35e2-4ad1-822d-f0157935b228 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:05.606731 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:05.607317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.607589 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.607870 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.608580 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:34:05.608874 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:05.610467 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:05.622832 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1430/2864] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:34:05.627274 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:05.637041 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacff28-7726-47fb-b66c-faa13a88149e None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:05.641743 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:05.642333 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.642609 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.642879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.643508 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:05.645778 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:05.784071 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1435/2865] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:34:05 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:05.788552 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:05.798701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a652c-4432-4000-8561-16fb8c67acf6 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:05.825105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:05.854904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:05.855503 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:05.855778 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:05.856076 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:05.856700 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:05.858601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.062814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1431/2866] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:34:05 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:06.067007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.080126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e17db-845c-442d-93f5-983f0e6076d4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.081983 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.082574 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.082854 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.083126 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.083790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.085406 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.095312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1436/2867] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:06.099305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.110377 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4867a-13c2-450f-b3ee-e393fdfbad1f admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.115443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.116753 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.116900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.117488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.117834 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.119465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e23dae-9a59-4926-8b3c-f988990c35c9 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.129730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1432/2868] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:06.133806 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.143787 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7d0-84e2-4570-b8a9-a6200d8427b6 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.148876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.149521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.149797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.150070 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] PATH_INFO: `/v3/projects/f132dd329fcb4ee8bc62aae3ab116138/users/4920b96d6ae04e58a6b03a57b419413b/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.165465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f132dd329fcb4ee8bc62aae3ab116138, user_id=4920b96d6ae04e58a6b03a57b419413b, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.168174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.269993 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1437/2869] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:06 2022] PUT /identity/v3/projects/f132dd329fcb4ee8bc62aae3ab116138/users/4920b96d6ae04e58a6b03a57b419413b/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:06.280919 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.281202 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.281474 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.309893 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] MFA Rules not processed for user `4920b96d6ae04e58a6b03a57b419413b`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:06.311096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:06.375925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1433/2870] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:06.380363 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.440699 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a20c-612e-440b-978e-86eae3d4f2d5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.445356 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.445950 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.446224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.446492 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.447135 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.449128 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.555698 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1438/2871] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:34:06 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:06.559827 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.571601 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76ea7-7200-43f8-8989-9a2a3965ada1 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.626760 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.627281 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.627558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.627883 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.628522 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.630425 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.765790 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1434/2872] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:34:06 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:06.772305 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.781268 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e406b-418c-47c0-af2b-08cf6d016ac5 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.785858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.786502 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.786779 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.787055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.787748 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.789334 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.801318 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1439/2873] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:06.805565 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.815846 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f6cee-e12d-41c2-830a-56731e71afef admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:06.821277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:06.821924 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:06.822205 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:06.822479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] PATH_INFO: `/v3/projects/751b2da71853482aa4c009f0318e59d5/users/6e29822ef2634aaab3a7ccb02979feb2/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:06.838646 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751b2da71853482aa4c009f0318e59d5, user_id=6e29822ef2634aaab3a7ccb02979feb2, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:06.841410 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:06.977189 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1435/2874] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:06 2022] PUT /identity/v3/projects/751b2da71853482aa4c009f0318e59d5/users/6e29822ef2634aaab3a7ccb02979feb2/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:34:06.981155 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:06.991880 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2018ef1-01ff-4051-9f7a-8ad50c382ebf admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:07.012269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:07.047367 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:07.048007 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:07.048289 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:07.048563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:07.049195 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:07.050839 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735209-245f-4d01-9003-4d7b880b8b4e admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:07.060859 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1440/2875] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:07.065444 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:07.071714 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:07.077113 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca1d19-e0cd-45a1-a60d-ae8f48e0fad7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:07.081943 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:07.082587 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:07.082863 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:07.083137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] PATH_INFO: `/v3/projects/751b2da71853482aa4c009f0318e59d5/users/6e29822ef2634aaab3a7ccb02979feb2/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:07.085160 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751b2da71853482aa4c009f0318e59d5, user_id=6e29822ef2634aaab3a7ccb02979feb2, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:07.087825 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:07.098139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1436/2876] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:07 2022] PUT /identity/v3/projects/751b2da71853482aa4c009f0318e59d5/users/6e29822ef2634aaab3a7ccb02979feb2/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:07.109650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:07.109935 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:07.110208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:07.139675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] MFA Rules not processed for user `6e29822ef2634aaab3a7ccb02979feb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:07.140917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:07.215776 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1441/2877] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:34:07.227262 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:07.228935 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:07.287810 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b240872-da5e-4d9d-bd7c-448b1aab5425 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:07.292039 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:07.292639 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:07.292918 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:07.293190 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:07.302885 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:07.304900 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:07.314300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1437/2878] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:07.545197 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:07.557374 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246014e5-b52f-4bb2-9332-ec14dd6dc4bf None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:07.560934 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:07.561559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:07.561833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:07.562101 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:07.619465 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:07.620994 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719965-5af6-4e20-820a-bb73b81e9913 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:07.630038 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1442/2879] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:11.648343 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:11.711804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ba05-3ea4-44e5-b03d-cae6ed11cb13 service cinder] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:11.716591 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:11.717218 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:11.717495 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:11.717788 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] PATH_INFO: `/v3/users/4920b96d6ae04e58a6b03a57b419413b` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:11.718944 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4920b96d6ae04e58a6b03a57b419413b)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:11.720904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:11.945047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4920b96d6ae04e58a6b03a57b419413b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.039549 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4920b96d6ae04e58a6b03a57b419413b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.040212 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4920b96d6ae04e58a6b03a57b419413b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.048080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4920b96d6ae04e58a6b03a57b419413b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.061469 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4920b96d6ae04e58a6b03a57b419413b'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.067490 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1438/2880] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:34:11 2022] DELETE /identity/v3/users/4920b96d6ae04e58a6b03a57b419413b => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:12.072219 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:12.072694 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:12.072797 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:12.092626 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:12.093861 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:12.135770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=100 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:12.154414 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1443/2881] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:12.164807 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:12.166217 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:12.181167 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:12.227316 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321954-83ea-4123-8581-336057b220d1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:12.231964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:12.232590 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:12.232876 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:12.233147 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:12.243568 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:12.245070 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:12.254925 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1439/2882] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:12.750117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:12.751578 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:12.765850 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b651a-6c3f-4dd6-bfc6-52b5add7d56b None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:12.770639 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:12.771277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:12.772204 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:12.772494 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] PATH_INFO: `/v3/projects/f132dd329fcb4ee8bc62aae3ab116138` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:12.773774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f132dd329fcb4ee8bc62aae3ab116138)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:12.775802 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:12.932321 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f132dd329fcb4ee8bc62aae3ab116138', 'request_id': 'req-7a8fcda2-9038-49a4-9be1-20ef011bdc56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.937879 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f132dd329fcb4ee8bc62aae3ab116138', 'request_id': 'req-7a8fcda2-9038-49a4-9be1-20ef011bdc56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:12.944831 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f132dd329fcb4ee8bc62aae3ab116138', 'request_id': 'req-7a8fcda2-9038-49a4-9be1-20ef011bdc56'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.045198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] The token cache is being invalidate because project f132dd329fcb4ee8bc62aae3ab116138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:34:13.045439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.046019 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.047653 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1444/2883] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:34:12 2022] DELETE /identity/v3/projects/f132dd329fcb4ee8bc62aae3ab116138 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:13.051730 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:13.062067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b2d9e-3844-4e8d-85ce-e9ecd25c7a5d service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:13.121277 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:13.122165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:13.122573 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:13.122911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] PATH_INFO: `/v3/users/6e29822ef2634aaab3a7ccb02979feb2` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:13.124186 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e29822ef2634aaab3a7ccb02979feb2)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:13.126192 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:13.351207 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e29822ef2634aaab3a7ccb02979feb2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.467131 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e29822ef2634aaab3a7ccb02979feb2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.467746 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e29822ef2634aaab3a7ccb02979feb2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.474928 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e29822ef2634aaab3a7ccb02979feb2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.487264 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-c9284e21-f34a-4cec-b20a-9fe5c8b879b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e29822ef2634aaab3a7ccb02979feb2'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:13.493526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1440/2884] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:34:13 2022] DELETE /identity/v3/users/6e29822ef2634aaab3a7ccb02979feb2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:14.041985 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:14.043419 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:14.058105 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fcda2-9038-49a4-9be1-20ef011bdc56 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:14.117245 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:14.117875 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.118156 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.118422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] PATH_INFO: `/v3/projects/751b2da71853482aa4c009f0318e59d5` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.119741 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] RBAC: Authorizing `identity:delete_project(project_id=751b2da71853482aa4c009f0318e59d5)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:14.121777 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:14.293312 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751b2da71853482aa4c009f0318e59d5', 'request_id': 'req-ce80b3eb-679d-4523-bda4-7748e323927b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:14.298814 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751b2da71853482aa4c009f0318e59d5', 'request_id': 'req-ce80b3eb-679d-4523-bda4-7748e323927b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:14.305650 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751b2da71853482aa4c009f0318e59d5', 'request_id': 'req-ce80b3eb-679d-4523-bda4-7748e323927b'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:14.400525 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:14.405960 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] The token cache is being invalidate because project 751b2da71853482aa4c009f0318e59d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:34:14.406298 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:14.407021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-ce80b3eb-679d-4523-bda4-7748e323927b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:14.408810 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1445/2885] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:34:14 2022] DELETE /identity/v3/projects/751b2da71853482aa4c009f0318e59d5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:14.414449 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.414939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.415052 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.435364 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:14.436605 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:14.465655 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:14.496843 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1441/2886] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:14.505571 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.505852 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.506124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.526264 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:14.527473 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:14.535929 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1446/2887] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:14.540752 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:14.553636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b2c0b-8b95-4781-bcf3-4835517aaa64 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:14.555228 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:14.555819 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.556130 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.556402 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.557067 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:34:14.557368 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:14.558991 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:14.570342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1442/2888] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:34:14.574457 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:14.584379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf91a2-18b8-405d-8b0c-73fdfd850f60 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:14.589421 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:14.590015 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.590293 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.590565 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.591206 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:14.593190 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:14.767240 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1447/2889] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:34:14 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:34:14.772198 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:14.782327 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3790d-8694-48b2-b1c2-691bba395ae2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:14.839194 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:14.839764 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:14.840066 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:14.840340 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:14.840966 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:14.842875 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:15.011176 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1443/2890] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:34:14 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:15.015340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:15.025886 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43c30b-3f12-4484-91d8-e31dded9512b admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:15.030518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:15.031123 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:15.031422 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:15.031700 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:15.032377 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:15.033963 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:15.047781 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1448/2891] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:15.056035 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:15.066065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19b61-6fb2-497c-8b81-9c6ad966f061 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:15.070547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:15.071139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:15.071418 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:15.071716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:15.072381 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:15.073969 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88b36-79fc-46f0-844d-ff95f3edfea2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:15.084426 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1444/2892] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:34:15.088796 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:15.098972 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c86bb-7a55-498a-a1fe-a2acd8f89106 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:15.104127 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:15.104774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:15.105055 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:15.105331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] PATH_INFO: `/v3/projects/ae6551875c72414790c88ede5d8378d4/users/b9de37d6b6144b2e904d908951d4a8d8/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:15.134025 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6551875c72414790c88ede5d8378d4, user_id=b9de37d6b6144b2e904d908951d4a8d8, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:15.136703 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:15.255396 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1449/2893] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:15 2022] PUT /identity/v3/projects/ae6551875c72414790c88ede5d8378d4/users/b9de37d6b6144b2e904d908951d4a8d8/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:34:15.272911 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:15.273195 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:15.273468 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:15.303342 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] MFA Rules not processed for user `b9de37d6b6144b2e904d908951d4a8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:15.313376 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:15.374603 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1445/2894] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:15.385975 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:15.387973 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:15.447277 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf9d6-eb83-4166-9f83-080811d1228f admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:15.450941 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:15.451545 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:15.451821 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:15.452121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:15.460984 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:15.462434 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d504fe-ef7f-4bf8-b0a2-4be51cb31ee4 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:15.471454 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1450/2895] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:34:19.863292 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:19.948886 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:19.983814 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a600d9-ca33-453b-b039-00976eaa6652 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:19.988751 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:19.989409 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:19.989690 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:19.990137 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] PATH_INFO: `/v3/users/b9de37d6b6144b2e904d908951d4a8d8` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:19.993220 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9de37d6b6144b2e904d908951d4a8d8)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:19.995308 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:20.230716 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9de37d6b6144b2e904d908951d4a8d8'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:20.319634 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9de37d6b6144b2e904d908951d4a8d8'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:20.320497 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9de37d6b6144b2e904d908951d4a8d8'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:20.327954 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9de37d6b6144b2e904d908951d4a8d8'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:20.339675 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9de37d6b6144b2e904d908951d4a8d8'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:20.345526 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1446/2896] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:34:19 2022] DELETE /identity/v3/users/b9de37d6b6144b2e904d908951d4a8d8 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:20.350470 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:20.350932 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:20.351032 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:20.370307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:20.371510 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:20.430790 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1451/2897] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:20.441459 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:20.442868 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:20.504733 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8ef2-8979-4c53-a6d6-a87d90fe1e01 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:20.505782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:20.506403 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:20.506701 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:20.507005 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:20.516286 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:20.517750 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9cd88-b5b3-4aa5-a835-3f4362b0dfe0 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:20.527860 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1447/2898] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:21.030537 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.031967 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:21.046205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a698d7-ecc7-4df7-b233-1b1820b27a26 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.051996 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.052631 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.052916 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.053187 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] PATH_INFO: `/v3/projects/ae6551875c72414790c88ede5d8378d4` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.054503 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6551875c72414790c88ede5d8378d4)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.056521 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.153858 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:21.165789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6551875c72414790c88ede5d8378d4', 'request_id': 'req-b8bea244-e338-4c36-8397-389280dd793c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:21.171205 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6551875c72414790c88ede5d8378d4', 'request_id': 'req-b8bea244-e338-4c36-8397-389280dd793c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:21.178144 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6551875c72414790c88ede5d8378d4', 'request_id': 'req-b8bea244-e338-4c36-8397-389280dd793c'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:21.261337 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] The token cache is being invalidate because project ae6551875c72414790c88ede5d8378d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:34:21.261681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:21.262258 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-b8bea244-e338-4c36-8397-389280dd793c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:34:21.263746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1452/2899] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:34:21 2022] DELETE /identity/v3/projects/ae6551875c72414790c88ede5d8378d4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:21.268673 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.269157 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.269299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.289430 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:21.290540 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:21.335313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:21.349196 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1448/2900] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:21.358792 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.359311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.359311 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.381024 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:21.389675 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:21.390278 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1453/2901] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:21.394525 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.404768 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b079709-9f36-4794-aa8b-8d6d2b138f1f None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.409329 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.409978 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.410254 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.410527 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.411175 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81891) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:34:21.411479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.413741 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.425355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1449/2902] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Aug 29 19:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:34:21.429663 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.438825 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eac03-65ee-4a1f-83f3-ad29f7b52555 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.443614 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.452169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.452169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.452169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.452169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.452169 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.545250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1454/2903] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:34:21 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:21.549506 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.560282 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993602e-7f32-4ec4-957c-f035c299d8c8 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.616429 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.624046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.624046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.624046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.624046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.624046 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.732008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1450/2904] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:34:21 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:21.736482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.747820 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7871e-66a2-4584-9117-6c5807311c97 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.752764 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.753364 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.753641 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.753914 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.754563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.756231 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.766081 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1455/2905] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:21.770141 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.780407 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb4393-0c46-455b-a4f0-9ff5d44ed2f2 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.785305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.785909 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.786188 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.786462 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.787692 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.789335 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c2fe1-ea4e-41a9-8589-4dfc63c630bc admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.799932 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1451/2906] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:21.804747 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:21.814876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b465-aba6-41bc-b5e1-c3a24bb796c3 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:21.819771 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:21.820447 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.820727 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.821000 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] PATH_INFO: `/v3/projects/f253ee21c29a44b4bfee93f14c26a2d5/users/a504697c696e4d8392adf0ae8adf98a7/roles/f1db7b15f40a4f39bce1393968346f60` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.835315 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f253ee21c29a44b4bfee93f14c26a2d5, user_id=a504697c696e4d8392adf0ae8adf98a7, role_id=f1db7b15f40a4f39bce1393968346f60)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:21.837971 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:21.948352 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1456/2907] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:21 2022] PUT /identity/v3/projects/f253ee21c29a44b4bfee93f14c26a2d5/users/a504697c696e4d8392adf0ae8adf98a7/roles/f1db7b15f40a4f39bce1393968346f60 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:21.953702 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308bae4-be2a-4981-b5b1-671834affb45 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:21.953986 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308bae4-be2a-4981-b5b1-671834affb45 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:21.954265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308bae4-be2a-4981-b5b1-671834affb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:21.975047 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-f308bae4-be2a-4981-b5b1-671834affb45 None None] MFA Rules not processed for user `417d5111f71945cab4c66bd4637fb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:21.976313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-f308bae4-be2a-4981-b5b1-671834affb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:22.038084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1452/2908] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:22.047706 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:22.049451 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:22.107511 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3c9f-333d-499f-9b9f-ff69685286e9 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:22.111259 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:22.111877 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:22.112168 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:22.112439 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:22.121917 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:22.123399 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:22.133201 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1457/2909] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:29.146114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.146114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.146114 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.178941 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] MFA Rules not processed for user `a504697c696e4d8392adf0ae8adf98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:29.180260 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:29.245976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:29.265354 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1453/2910] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:29.269870 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.279987 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b039d-6f2c-431f-b04a-8e48d2804bb9 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.284426 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.285017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.285290 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.285559 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.286174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.288588 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.452501 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1458/2911] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Aug 29 19:34:29 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:29.456559 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.468222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df668342-ed81-43d1-964c-ab3f694ce8ee None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.530972 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.531676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.532080 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.532422 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.533124 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.535128 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.634443 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1454/2912] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Aug 29 19:34:29 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:34:29.638563 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.649618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbebd7-e3f7-4dae-8828-3f00281581c4 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.654441 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.655067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.655348 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.655655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.656695 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.658339 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.673624 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1459/2913] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:29.675313 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.685669 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937701c-57a1-4745-b719-855841016fd5 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.690328 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.690979 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.691274 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.691558 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] PATH_INFO: `/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468/users/b2ed4224ebbc4cf0b17ccd29dc536115/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.706994 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86d67f8d1244d3e9e46ce9961b7b468, user_id=b2ed4224ebbc4cf0b17ccd29dc536115, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.710300 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.810650 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1455/2914] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:29 2022] PUT /identity/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468/users/b2ed4224ebbc4cf0b17ccd29dc536115/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:29.815247 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.826068 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa3701-f148-4d3f-a72a-1e37df3a2afa admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.851320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:29.889518 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.890131 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.890407 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.890676 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] PATH_INFO: `/v3/roles` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.891340 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.892965 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a86fac-2413-418d-8a17-f0851fe2878c admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.909117 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1460/2915] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Aug 29 19:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:34:29.910471 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:29.921383 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c144-640f-4f91-9669-e893f588f4e7 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:29.934727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:29.934727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:29.934727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:29.934727 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] PATH_INFO: `/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468/users/b2ed4224ebbc4cf0b17ccd29dc536115/roles/ae077013d56245c7869e76158c81fd53` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:29.935563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86d67f8d1244d3e9e46ce9961b7b468, user_id=b2ed4224ebbc4cf0b17ccd29dc536115, role_id=ae077013d56245c7869e76158c81fd53)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:29.935563 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:29.945075 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1456/2916] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Aug 29 19:34:29 2022] PUT /identity/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468/users/b2ed4224ebbc4cf0b17ccd29dc536115/roles/ae077013d56245c7869e76158c81fd53 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:34:36.217752 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:36.218243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:36.218613 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:36.251404 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] MFA Rules not processed for user `b2ed4224ebbc4cf0b17ccd29dc536115`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:36.252594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:36.351272 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1461/2917] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:36.374061 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:36.377795 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:36.469904 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739e132-7a81-42f7-aef0-6f86d434fed0 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:36.476008 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:36.476640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:36.476895 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:36.477165 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:36.514142 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:36.538288 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:36.539975 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f2b9b-10b2-4778-86aa-09f201a9785d service nova] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:36.549545 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1457/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:46.344307 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=51 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:46.856888 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=11 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:34:52.356833 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:52.358296 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:52.437774 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf504f91-510a-406e-afe3-48079eae53e6 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:52.443908 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:52.444551 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:52.444826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:52.445100 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:52.446887 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:52.461017 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:52.462638 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:52.473594 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1462/2919] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:34:57.486516 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:57.487105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:57.487105 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:57.508691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] MFA Rules not processed for user `a504697c696e4d8392adf0ae8adf98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:34:57.511470 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:57.517159 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1458/2920] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:34:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:34:57.866094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:34:57.869809 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:57.927228 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44221227-1a43-4035-a814-a777bc0c3e24 service placement] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:34:57.931172 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:34:57.931815 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:34:57.932116 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:34:57.932390 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:34:57.934132 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:34:57.944323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:34:57.945813 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:34:57.954428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1463/2921] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:34:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:35:00.512929 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:35:00.525170 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd1a30-d77d-4b76-91ce-e7184142d1bb None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:35:00.530209 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:35:00.530902 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:35:00.531248 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:35:00.531547 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:35:00.533465 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:35:00.546265 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:35:00.553628 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:35:00.559097 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1459/2922] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:35:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:35:07.951124 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=58 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:35:10.554569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=9 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:35:23.051094 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:35:23.066214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d470-a935-48ff-951b-01cc23b55e96 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:35:23.073383 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:35:23.074047 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:35:23.074385 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:35:23.074738 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:35:23.076482 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:35:23.090145 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:35:23.091798 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:35:23.103087 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1464/2923] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:35:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:35:26.514315 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:35:26.524399 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa44c-e09d-4542-b853-fb8b2dec382f service placement] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:35:26.530981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:35:26.530981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:35:26.530981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:35:26.530981 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:35:26.541724 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:35:26.543173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:35:26.554091 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1460/2924] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:35:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:35:28.942548 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:35:28.944250 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:35:28.961072 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f473-c851-413a-9b8b-435cd8d9cd48 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:35:28.967323 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:35:28.968222 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:35:28.968841 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:35:28.969179 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:35:28.972384 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:35:28.990060 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:35:28.993230 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:35:29.010791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1465/2925] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:35:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:35:36.549007 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=6 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:35:39.001684 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=12 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:36:49.009276 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:36:49.018393 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0c8b7-dfdc-4d6c-aa01-cd4a93dd12f0 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:36:49.022136 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:36:49.022728 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:36:49.023006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:36:49.023275 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:36:49.025129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:36:49.034945 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:36:49.036411 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:36:49.044695 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1461/2926] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:36:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:36:59.040857 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=6 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:37:24.588618 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:37:24.627097 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec445-473b-4c76-ad6c-bbb7bc561f11 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:37:24.630826 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:37:24.631428 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:37:24.631702 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:37:24.632011 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:37:24.633705 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:37:24.678696 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:37:24.680160 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:37:24.691872 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1466/2927] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:37:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:37:25.875006 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:37:25.913107 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d648f-7a05-4f1c-bab0-87484c415700 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:37:25.916933 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:37:25.917536 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:37:25.917813 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:37:25.918084 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:37:25.919939 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:37:25.934584 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:37:25.936222 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ffeec-5f29-47eb-8c17-1e9316e75f05 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:37:25.947729 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1462/2928] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:37:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:37:27.400868 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:37:27.414224 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e6f69-24a9-4ab7-b5fa-78ef1a65f511 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:37:27.428236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0af6b468c7442e818efec1543b7eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:37:27.428236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:37:27.428236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:37:27.428236 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:37:27.428876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:37:27.439139 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:37:27.440977 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:37:27.453620 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1467/2929] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:37:29.382588 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] REQUEST_METHOD: `POST` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:37:29.383117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:37:29.383117 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:37:29.403317 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.auth.core [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] MFA Rules not processed for user `a504697c696e4d8392adf0ae8adf98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81891) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:37:29.404830 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:37:29.411931 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1463/2930] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:37:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:37:37.448218 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=37 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:37:39.397299 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=21 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:39:46.531370 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:39:46.532784 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:39:46.611021 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bffea1-a9ae-4581-a4aa-68be6c0acf69 service nova] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:39:46.614647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb59243400b44b8e998daaa4bd8f1ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:39:46.615243 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:39:46.615519 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:39:46.615785 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:39:46.624598 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:39:46.626016 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b7c84-3494-46a2-9610-82fa207c0228 service cinder] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:39:46.634647 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1468/2931] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:39:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:39:55.330676 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:39:55.332179 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:39:55.370748 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc350746-20f1-4bca-a81e-0d00f3463728 None None] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:39:55.375923 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709099d-9712-45fb-8456-14019abe283b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:39:55.383096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709099d-9712-45fb-8456-14019abe283b service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:39:55.383096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709099d-9712-45fb-8456-14019abe283b service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:39:55.383096 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709099d-9712-45fb-8456-14019abe283b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:39:55.386804 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709099d-9712-45fb-8456-14019abe283b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:39:55.388488 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709099d-9712-45fb-8456-14019abe283b service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:39:55.399126 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1464/2932] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:39:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:39:56.630740 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=35 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:39:56.786655 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] REQUEST_METHOD: `POST` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:39:56.786981 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:39:56.787320 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:39:56.809496 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.auth.core [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] MFA Rules not processed for user `a504697c696e4d8392adf0ae8adf98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81890) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:39:56.813067 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:39:56.818562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1469/2933] 10.222.0.62 () {58 vars in 958 bytes} [Mon Aug 29 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:39:57.162873 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709099d-9712-45fb-8456-14019abe283b service placement] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:39:57.173339 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709099d-9712-45fb-8456-14019abe283b service placement] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:39:57.177024 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:39:57.177636 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:39:57.177910 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:39:57.178178 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:39:57.179983 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:39:57.204865 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:39:57.206330 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:39:57.215569 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1465/2934] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:39:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:40:06.803930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=3 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:40:07.210976 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=26 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:41:53.044907 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:41:53.055721 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5ac5-68f0-427b-b37e-301701622ad8 None None] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:41:53.059269 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:41:53.059876 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:41:53.060159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:41:53.060424 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:41:53.067995 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:41:53.069430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:41:53.077644 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1470/2935] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:41:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:42:03.073978 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=6 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:42:26.838653 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:26.848213 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600627-0ff1-4572-be3a-8ee6207da80a service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:26.852139 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f1f1b96cd6477581e1040d8d8e0529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:26.852786 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:26.853065 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:26.853382 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:26.928735 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:26.933305 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:26.952268 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1466/2936] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:42:30.930198 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:30.944599 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae738196-91be-48dc-af10-4c6a8b9ce0c8 service neutron] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:30.949773 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:30.950387 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] REQUEST_METHOD: `GET` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:30.950658 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:30.950930 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:30.952766 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:42:30.965214 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:30.967891 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:30.980310 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1471/2937] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:42:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:42:35.453763 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:35.463767 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf585-e7dd-4dc0-b395-6d013404a461 service neutron] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:35.468458 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:35.469095 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:35.469369 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:35.469640 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] PATH_INFO: `/v3/users/a504697c696e4d8392adf0ae8adf98a7` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:35.470828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a504697c696e4d8392adf0ae8adf98a7)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:35.472828 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:35.776129 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a504697c696e4d8392adf0ae8adf98a7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:35.885326 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a504697c696e4d8392adf0ae8adf98a7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:35.885964 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a504697c696e4d8392adf0ae8adf98a7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:35.893891 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a504697c696e4d8392adf0ae8adf98a7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:35.906259 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a504697c696e4d8392adf0ae8adf98a7'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:35.925641 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1467/2938] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:42:35 2022] DELETE /identity/v3/users/a504697c696e4d8392adf0ae8adf98a7 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:42:36.237654 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:36.249584 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07425dd-6a5f-4de9-9447-6117664459af service placement] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:36.320235 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:36.320770 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:36.321049 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:36.321331 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] PATH_INFO: `/v3/projects/f253ee21c29a44b4bfee93f14c26a2d5` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:36.322477 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f253ee21c29a44b4bfee93f14c26a2d5)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:36.324430 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:36.449138 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f253ee21c29a44b4bfee93f14c26a2d5', 'request_id': 'req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:36.455746 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f253ee21c29a44b4bfee93f14c26a2d5', 'request_id': 'req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:36.463159 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f253ee21c29a44b4bfee93f14c26a2d5', 'request_id': 'req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:36.602174 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] The token cache is being invalidate because project f253ee21c29a44b4bfee93f14c26a2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:42:36.602429 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:36.603002 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:36.607681 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1472/2939] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:42:36 2022] DELETE /identity/v3/projects/f253ee21c29a44b4bfee93f14c26a2d5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:42:36.609185 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:36.610691 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:42:36.624355 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2667-026c-4336-b68d-fbdf5053bd7a admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:36.674026 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:42:36.686140 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:36.686780 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:36.687058 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:36.687331 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] PATH_INFO: `/v3/users/b2ed4224ebbc4cf0b17ccd29dc536115` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:36.688491 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ed4224ebbc4cf0b17ccd29dc536115)` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:36.691019 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:36.957684 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ed4224ebbc4cf0b17ccd29dc536115'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.058173 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ed4224ebbc4cf0b17ccd29dc536115'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.058884 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ed4224ebbc4cf0b17ccd29dc536115'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.066982 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ed4224ebbc4cf0b17ccd29dc536115'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.079017 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.notifications [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ed4224ebbc4cf0b17ccd29dc536115'} {{(pid=81891) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.085389 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1468/2940] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Aug 29 19:42:36 2022] DELETE /identity/v3/users/b2ed4224ebbc4cf0b17ccd29dc536115 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:42:37.363379 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Authenticating user token {{(pid=81890) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:42:37.365791 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.fernet_utils [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81890) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:42:37.379562 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b0a3-dfc5-4423-8a20-6641c2e74ed0 admin admin] Validating token access rules against request {{(pid=81890) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:42:37.445512 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417d5111f71945cab4c66bd4637fb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb936e104dc24a75a45f4d04495d191b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81890) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:42:37.446173 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:42:37.446467 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:42:37.446736 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] PATH_INFO: `/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468` {{(pid=81890) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:42:37.447901 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86d67f8d1244d3e9e46ce9961b7b468)` {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:42:37.450051 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] RBAC: Authorization granted {{(pid=81890) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:42:37.461958 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:42:37.591147 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86d67f8d1244d3e9e46ce9961b7b468', 'request_id': 'req-67931b93-b91f-4a5c-952b-9d117274c3d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.597892 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86d67f8d1244d3e9e46ce9961b7b468', 'request_id': 'req-67931b93-b91f-4a5c-952b-9d117274c3d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.604789 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86d67f8d1244d3e9e46ce9961b7b468', 'request_id': 'req-67931b93-b91f-4a5c-952b-9d117274c3d5'} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.777885 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] The token cache is being invalidate because project c86d67f8d1244d3e9e46ce9961b7b468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81890) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:42:37.778121 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.778783 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG keystone.notifications [None req-67931b93-b91f-4a5c-952b-9d117274c3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81890) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:42:37.780478 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: [pid: 81890|app: 0|req: 1473/2941] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Aug 29 19:42:37 2022] DELETE /identity/v3/projects/c86d67f8d1244d3e9e46ce9961b7b468 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:42:47.623881 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:42:48.058208 dv-cinder-855013-1-iscsi devstack@keystone.service[81890]: DEBUG dbcounter [-] [81890] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81890) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:45:02.312031 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Authenticating user token {{(pid=81891) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:45:02.313155 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:45:02.390999 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe10-d2ba-48a2-a505-1ea2eed20bd1 admin admin] Validating token access rules against request {{(pid=81891) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:45:02.451815 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd6962ec6f5425091a001e8ee251616', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69c75d1e1ab2498f95e156d48272226d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81891) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:45:02.452479 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] REQUEST_METHOD: `GET` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:45:02.452756 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] SCRIPT_NAME: `/identity` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:45:02.453030 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81891) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:45:02.454782 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.fernet_utils [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81891) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:45:02.467657 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:45:02.469322 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6efa96-8a25-40dd-9ffe-87a8d34992ca service placement] RBAC: Authorization granted {{(pid=81891) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:45:02.480848 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: [pid: 81891|app: 0|req: 1469/2942] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Aug 29 19:45:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:45:12.475948 dv-cinder-855013-1-iscsi devstack@keystone.service[81891]: DEBUG dbcounter [-] [81891] Writing DB stats keystone:SELECT=58 {{(pid=81891) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}